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Introduction 



A dynamical system which changes state at discrete points in time is called a 
discrete event dynamic system. The theory of discrete event dynamic systems 
is mainly used in the study of manufacturing systems, telecommunication 
networks and transportation systems. 

The development of this theory has undergone a dramatic change of per- 
spective recently thanks to the introduction of general principles which are 
useful to a wide range of application domains. Such principles include the 
use of the (max, plus) algebra [23] and more generally topical functions [57] 
to model the synchronizations present in the system as well as the network 
calculus [39, 37]. Another example is the work of [52, 53] where rather ele- 
mentary monotone structures are used to derive powerful results. 

The approach adopted here goes in the same direction. This book will 
introduce several general principles useful in the control of discrete event 
systems. 

The aim of this monograph is not to offer a complete theory of discrete 
event control of stochastic networks, but to derive a theory and applications 
based on multimodularity and regularity. The main objective is to show that 
for a large class of stochastic discrete event systems and under rather natural 
assumptions on the behavior of the system as well as on the stochastic pro- 
cesses driving its evolution, the smoother the input process, the better the 
performances of the system. 

Of course, the notions smoothness and the performance criteria have to be 
made precise. This requires several technicalities using several notions from 
convex analysis, stochastic processes and word combinatorics. However, this 
must not hide the general underlying goal of the whole work. 

This book is focused on a wide class of control (or of optimization) prob- 
lems over sequences of integer numbers. We know that the theory of convex 
functions plays a key role in the theory of optimization over convex spaces. 
An important objective is to construct a counterpart for our setting in which 
the optimization is not done any more over a convex set. A natural candi- 
date to replace a convex function over some set Z n of n-dimensional vectors 
of integers is an integer-convex function, i.e. a real valued function / that 
satisfies the standard convexity condition 

f(ax + (1 - a)y) < af(x) + (1 - a)f(y), 
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for all x,y £ Z n and for all a £ (0,1) for which ax + (1 — a)y £ Z n . 
Unfortunately, this intuitive counterpart of convexity turns out to be too 
restrictive for our purpose. In particular, it does not even guarantee that a 
local minimum is a global minimum! 

The natural counterpart of convex functions over integer sets turns out to 
be the so called multimodular functions, introduced in [59] , and for such func- 
tions we have indeed the property that local minima are global minima. The 
property of multimodularity turns out to be useful in a much more general 
context: the control of discrete event systems. As we shall illustrate through- 
out this monograph, the natural performance measures in many problems 
in queuing, in telecommunications and in other areas of applications turn 
out to be multimodular. This includes many admission control problems to 
networks, routing control into networks, service assignment problems and 
control of vacations. Typical performance measures that are multimodular in 
these problems are expected waiting times, sojourn times and queue lengths. 
For all these problems, it turns out that the multimodularity of the costs 
induces, in many case, a particular form of optimal policies. They turn out 
to be very " regular" , and can be described by the well known " bracket" se- 
quences. Thus the study of multimodular functions is strongly related to the 
study of bracket policies. 

The definition of multimodularity goes back to the seminal paper by Ha- 
jek [59], who introduced this term in order to study a problem of optimal 
admission control into a single queue under no queue information. The pre- 
cise problem was to admit customers to a single queue, under the constraint 
that the long run fraction of customers admitted be at least p. The optimality 
of a policy based on a bracket sequence of admission actions was obtained 
in [59] for the number of customers in a one-server queue with exponential 
service and a renewal arrival process. 

Another application of multimodular functions is in the control of queues 
with full state information. Weber and Stidham [114] and Glasserman and 
Yao [52, 53] obtained monotone properties of the optimal control policies as 
a function of the state, in a variety of queueing control and related problems. 
The methodology was strongly based on the multimodularity properties of 
the immediate costs and the cost-to-go functions. 

In this book, we develop mainly the tools for control problems with no 
state information. This is done both in a deterministic setting as well as in 
a very general stochastic framework. Two classes of problems are handled. 
In the first one, the control sequence is one dimensional; it covers admission 
control, service assignment and vacation control problems. Although the con- 
trol is one dimensional, the systems to which it is applied to may be quite 
general and complex. We focus on general discrete event models which can 
be described as linear in the max-plus algebra. This type of problem is fully 
solved, and an optimal policy is identified. A second type of problems we 
handle is the one in which the control is multi-dimensional. It covers rout- 
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ing as well as polling problems. Optimal policies are obtained only in special 
cases: the case of symmetrical models, systems with dimension two and many 
others. The restriction is due to the fact that the regularity properties that 
characterize optimal policies in dimension one cannot be generalized to di- 
mension greater than two, except for very special cases. The identification of 
all cases in which regularity is possible in dimension larger than two has been 
an open challenging problem since several decades since it was formulated 
in the well known Fraenkel conjecture [46], and it prompted much research 
[55, 91, 107, 115, 93, 46, 90]. From the cases of low-dimension (up to six) 
which have been fully solved [108] we know that sequences that are "regu- 
lar" in all components are very rare. We therefore need other tools to handle 
higher dimensions. This motivates us to consider the question of regular or- 
dering between policies, which basically aims at identifying orders between 
policies such that if a policy is greater than another in that ordering then 
it yields a better cost. We also derive bounds on the performance measures 
by using a new combinatorial notion, called unbalance of a multidimensional 
control sequence. 

Although our main concern in the book is to handling control problems 
with no information, we also investigate the problem of closed-loop control in 
which the control has full or partial state information. We develop a general 
framework for handling multimodular cost functions, and we establish the op- 
timality of optimal monotone policies: these are of threshold type for the one 
dimensional case, and of a switching-curve type for the two-dimensional case. 

This book is structured in four parts. 

Part I: Theoretical foundations, presents the theoretical foundations 
which consists of three notions: multimodularity, balanced words, bracket 
sequences and stochastic Petri nets. The material in Part I is used through- 
out the monograph. The other chapters can be studied independently. 

In Chapter 1 we present the basic definition of multimodularity and inves- 
tigate the properties of multimodular functions. Using these properties, we 
obtain general optimization results which are in particular useful for average 
cost criteria in open-loop control. We show in that context that the expected 
average cost problem is optimized by using bracket sequences. 

Chapter 2 introduces the balanced sequences, the Sturmian words and 
bracket sequences and shows the close relations existing between them. It 
also details several ways to construct such sequences as well as some of their 
properties, useful in this context. More details on the combinatorial properties 
of this sequences can be found in [84] . 

In order to apply the tools developed in Chapters 1 and 2 to networks, and 
more generally to discrete event dynamic systems, we present in Chapter 3 the 
formalism of Petri nets, and more precisely of the so called stochastic event 
graphs; These are systems whose dynamics follows a linear vectorial evolu- 
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tion equation in the so-called (max,+) algebra, and it covers many queuing 
networks. 

Part II: Admission and routing control, covers the application of the op- 
timisation theorems for optimal admission and routing control in discrete 
event stochastic systems. 

Using the above formalism we consider in Chapter 4 the first application 
of the general theory to problems of admission control into networks, where 
we show the main theorems. We show that open loop admission control in 
a stochastic network with (max,plus) dynamics is optimal for the traveling 
time of customers when the routing policy is Sturmian. 

Chapter 5 discusses the relevance of the assumptions made so far for 
networks of queues, and in particular, the issue of cross traffic. 

The objective pursued in Chapter 6 is to generalize the admission control 
problems to the case of routing control. The first part addresses the following 
combinatorial problem: is it possible to construct an infinite sequence over K 
letters where each letter is distributed as "evenly" as possible and appears 
with a given rate? The second objective of the Chapter is to use this con- 
struction in the framework of optimal routing in queuing networks. We show 
under rather general assumptions that the optimal deterministic routing in 
stochastic event graphs is such a sequence. 

While Chapter 6 says that a bracket squence is optimal for routing into 
two parallel systems, it does not give any hint on the actual optimal policy. 
Chapter 7 gives the computation of the optimal policy in the simple case of 
deterministic queues. It is rather surprising that this computation uses the 
decomposition in continuous fraction of the parameters of the system. It may 
also be surprising that greedy policies such as "join the shortest queue" are 
not always optimal here. 

Part III: Several extensions, shows how the previous results can be useful 
in other cases. 

In Chapter 8 we consider the problem of optimal routing of arriving pack- 
ets into K servers having no waiting room. Packets that are routed to a busy 
server are lost. We consider two problems where the objective is to maximize 
the expected throughput (or equivalently, minimize the loss rate). We assume 
that the controller has no information on the state of the server. We establish 
the optimality of the bracket sequences, for exponential service times and gen- 
eral stationary arrival processes, which include, in particular, the interrupted 
Poisson process, Markov modulated Poisson Process (MMPP) and Markov 
arrival process (MAP). Based on this solution, we solve the dual problem of 
optimal assignment of a single server to several single server queues to which 
packets arrive according to Poisson processes. A first step for the solution 
of this problem is to establish the multimodularity properties of the costs, 
which together with some other conditions that we establish, imply the reg- 
ularity of optimal policies. We go however beyond this characterization of 
optimal policies and show that (i) there exist optimal periodic policies; this 
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is established using some tools from Markov decision processes; (ii) Policies 
that are regular in a weaker sense (related to the majorization ordering) are 
also optimal; this is done using the Schur convexity ordering. We finally apply 
our framework to the problem of robot scheduling for web search engines. 

In Chapter 9 we consider the optimal open-loop control of vacations in 
queuing systems. The controller has to take actions without state informa- 
tion. We first consider the case of a single queue, in which the question is 
when should vacations be taken so as to minimize, in some general sense, 
workloads and waiting times. We then consider the case of several queues, 
in which service of one queue constitutes a vacation for others. This is the 
optimal polling problem. We solve both problems using concepts from mul- 
timodularity. 

Chapter 10 shows how multimodularity can be used for closed loop con- 
trol. Using this approach, we show that the optimal policy has monotonicity 
properties for a rather general class of problems. 

Part IV: Comparisons, derives the theory of routing to three queues or 
more. Since the structure of the optimal policy is unknown in this case, the 
analysis is focused on getting lower and upper bounds for the performance 
measures, and on the comparison of the systems for different admisssion 
sequences. 

Chapter 11 shows how the traveling times in a FIFO-stochastic event 
graph are compared in increasing convex ordering. Two comparison lemmas 
are proved. As application of the first one, we derive that: 

— Independent sources perform better than coupled sources. 

— Fixed batch sizes are better than randon batch sizes. 

— Fluid scaling improves the performance. 

The second comparison lemma implies a lower bound on the traveling time. 

In Chapter 12, the notion of multimodularity is generalized to sim- 
plex convexity. Whereas multimodular functions are defined through the L- 
triangulation, simplex convexity is defined on any triangulation. It turns out 
that the theory of Chapter 1 can be generalized to simplex convexity and 
the notion of cone ordering is introduced. It is related to the "distance" of a 
given sequence to the sequence minimizing a given simplex convex function. 
An application is made for periodic admission sequences in queues. 

Chapter 13 introduces several other orders (graph order, unbalance) which 
are compared with the cone ordering. These new ways to compare sequences 
are used to derive bounds on the average waiting time in queues for several 
arrival processes. 

Finally, Chapter 14 shows the link existing between regularity (in the 
multimodular sense) and majorization (in the Schur convex sense) and how 
this can be used in queues. The link between the two notion is given by the 
notion of regular perserving functions (which is stronger than multomodu- 
larity). It can be used to compare the maximum waiting time in stationary 
systems rather than the average waiting time, as for multimodularity. 
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Finally, note that the notation is uniformized wherever possible. However, 
the chapters in Parts II, III and IV are rather independent from each other 
and notations may sometimes vary slightly between different chapter. Most 
chapters are based on material which is published in journal papers ([6, 4, 5, 
3]). We are gratefulto the publishers of these journals for their permission to 
include adaptations of these texts in this monograph. 
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The first part of the book is made of three chapters, dedicated respectively 
to the notions of multimodularity, balanced sequences (also called Sturmian 
words) and Petri nets. 

In the following, we will mainly deal with discrete event systems modeled 
as Petri net with an input process controlled by Sturmian words with a cost 
function which is multimodular. 

The three notions are used together in two general theorems: Theorem 7 
which states that Cesaro limits of multimodular functions are minimized by 
Sturmian sequences and 18 which shows that the average traveling time of 
tokens in a Petri net is a multimodular function of the input process. 



1 Multimodularity, Convexity and 
Optimization 



1.1 Introduction 

This chapter presents the theoretical framework useful in the following chap- 
ters dedicated to the control of the input in networks. It is mainly focused on 
the study of multimodular functions which can be seen as a discrete analog 
to convex functions. 

We provide elementary proofs for properties already established by Hajek, 
who showed that the lower convex envelope of a multimodular function / is 
the piecewise linear interpolation on a specific triangulation of the space by 
simplices called atoms. In this chapter, we show directly that this linear in- 
terpolation is convex if and only if / is multimodular. A more general version 
of this result will also be presented in Chapter 12. This allows us to restrict 
the study of multimodular functions to convex subsets of Z m which are con- 
vex unions of atoms (or faces of atoms). Additional interesting properties of 
multimodular functions are also presented. 

In the second part of this chapter, we develop basic optimization tools for 
average costs. We establish lower bounds for average costs using Abel-type 
asymptotic techniques. We also show that the lower bounds are achieved by 
bracket sequences. Such costs depend on a sequence of multimodular func- 
tions, rather than on a single multimodular function. This is a nice feature of 
our approach since the optimization results can be applied directly to average 
costs as long as the assumptions used in Theorems 6 or 7 are satisfied. This 
is not the case when a single cost function is used, as in [59], where for any 
specific application, additional analysis has to be done before one may apply 
the general minimization results to average costs problems. 

We illustrate the usefulness of this theory in admission control into a very 
simple queue and we provide a detailed analysis of the D/D/l queue with 
fixed batch arrivals, with no state information. We show that the policy which 
is defined through a balanced sequence minimizes the average queue length for 
the case of an infinite queue, but not for the case of a finite buffer. However, 
when further restricting to those policies for which no losses occur, we obtain 
again the optimality of balanced policies. To conclude that example, we study 
also the case where it is possible to admit a part of an arriving batch. 

The more general cases such as the G/G/l queue and networks of queues 
are treated in Chapter 4. 
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1.1.1 Organization of the chapter 

In the first part of the chapter, we develop basic optimization tools for average 
costs. We establish lower bounds for average costs using Abel- type asymptotic 
techniques. 

In follow-up chapters, we shall make use of all the theoretical results of this 
chapter in order to study more general admission and service control problems 
in dynamic systems that can be described using the max-plus algebra, with 
general stationary inter-arrival and service times. 

1.2 Properties of multimodular functions 

We present in this section a short overview of Hajek's theory of multimodular 
functions. Some additional results are also established. We begin by present- 
ing the definition of multimodularity, and some general properties (Subsection 
1.2.1) which are interesting by their own and that will be used in subsequent 
work. We then present in Subsection 1.2.2 the relation between multimodu- 
larity and convexity. The properties presented in Subsection 1.2.2 are those 
needed in the following sections and subsequent chapters on optimization and 
control. 

In the following, Z will denote the set of integer numbers and N the set 
of non- negative integer numbers, N = {0, 1, • • • }. 

Let ei £ W 1 , i = 1, ■ ■ ■ , m denote the vector having all entries zero except 
for a 1 in its ith entry. Define Si = e^_i — e,, i = 2, - ■ ■ ,m (for an integer 
i taking values between 1 and m, we understand throughout i — 1 = m for 
i = 1). 



Let T = {-ei, s 2 , ■ ■ 



i $m i 


e m }, where 








-ei = 


(-10 ■■ 


• 





) 


S2 = 


(1-10- 


• 





) 


S3 = 


( 1 -1 ■■ 


• 





) 



s m = ( ••• 1 -1 ), 
e m = ( ••• 1 ). 

T will be called a multimodular base of Z m . 

In a typical application, an element x € W 1 will be used to denote a con- 
trol sequence; in admission control problems Xk would then have the meaning 
of the number of customers accepted to a system at time slot k. The vector 
x + e m then has the meaning of admitting one more customer at the last 
epoch (as compared to the acceptance pattern of a vector x) , whereas x + sj 
has the meaning of "shifting" a customer to the "left", i.e. accepting one 
customer less at the jth slot and one more at the (j — l)st slot (with respect 
to the acceptance pattern given by x). 
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Define T = {ei, —ei, Si, —s,, i = !,■■■ , to}, and T = {—it : u £ J 7 }. 
Let M be the set of all real numbers. 

Definition 1 (Hajek). A real-valued function f : Z m — ► M is multimod- 
ular with respect to T if for all x £ Z™ , v and w in T , v ^ w, the following 
holds: 

f(x + v)+ f[x + w)> f{x) + fix + v + w). (1.1) 

Remark 1. A function / on Z m is multimodular with respect to T if and 
only if / is multimodular with respect to T. This can be checked easily using 
y = x + v + w instead of x in Equation 1.1. 

Unless otherwise stated, we shall say that / is multimodular if it is mul- 
timodular with respect to T . A more general definition of multimodularity 
wll be given in Chapter 12, where the base may not be T or T . 

1.2.1 General properties 

For a function g defined on Z™, define 

Aigix) = A ei gix) = gix + ei) - gix) and A Si g = A^ig - Aig. 

We further define A- ei g = gix — ei) — gix). Note that A Si gix) = gix + ei + 
Si) - gix + ei). 

It is easy to check that 

Lemma 1. A v is a linear operator for any v € T (i.e. A v iug + \f) = 
uA v g + XA v f). For all v,w e T, A v A w g = A w A v g. 

Lemma 2. (a) f is multimodular if and only if 

A v A w f<0 (1.2) 

for all v, w £ T , w ^ v. 
(b) If f is multimodular then 



— (b.i) For any w € T 

- (b.ii) For all 

— (b.iii) For all 

- (b.iv) A ei A s J>0. 



with w^ei. 

A ei A w f > 0. 

AiAjf > 0. 
A,A : .f > AiAjf. 



(1.3) 

(1.4) 
(1.5) 
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- (b.v) A ei A Bj f <0,j<i and A ei A Bj f >0,j> i. 

- (b.vi) A S1 A S J<0, 

- (b.vii) A Si A s J > 0. 

(c) Consider the 2- dimensional case: T = {— ei, S2, ei\- Assume that (1-4) 
and (1.5) hold. Then f is multimodular. 

The proof of the lemma is technical and tedious. It is given in an appendix 
(Section 1.7). 

Note that Equations (b.ii) and (b.iii) can be seen as a discrete counterpart 
of the characterization of convexity using second derivatives in the continuous 
domain. Equations (b.iv), (b.v), (b.vi) and (b.vii) are useful when dealing 
with functions of multimodular functions, like projections. Indeed, checking 
multimodularity of projections or restrictions of multimodular functions is 
easier using this approach. 

For example, given a point X in Z™, a set I of indices i\ < ■ ■ ■ < ik 
with k <n and the function px,i ■ % k — ► Z m defined by px,i{Y) = Z, with 
Zi = Xi if i I and Z^ = Yj otherwise, we have the following property. 

Lemma 3. If f : Z m — > R is multimodular then for all I and X , the function 
g : Z k — ► R defined by g{Y) = f{px,i{Y)) is multimodular. 

Proof. The proof holds by checking that relations (1.2) holds for g. We will 
denote by T k = {— e\ , s\, ■ ■ ■ s%, e£} the multimodular base of Z fc . The base 
of Z™ is T. Using the previous notations, we have to check (1.2) for g with 
respect to T k . 

A_ ek A sk g(Y)=A_ eii A ij f(Z) 
= £ A_ eii A s J(Z) 

t=ij-i+l 

<o, 

by Equation (b.v). 

The other cases, A s kA s kg(Y) and A s kA e kg(Y) are checked similarly, rj 

1.2.2 Multimodularity and convexity 

In the first part of this section, we present some details on the construction 
of a simplicial decomposition of t m adapted to the base of multimodular- 
ity. This construction was given by Hajek. Theorem 4.3 in [59] proves that 
the lower convex envelope / of a multimodular function / is the piecewise 
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linear interpolation of / on this triangulation. In this monograph we define 
/ directly as the piecewise linear interpolation of /. The second part of the 
section is then devoted to Theorem 1, that shows that / is multimodular if 
and only if / is convex, and some of its consequences which are useful to 
derive optimality results (see Section 5). 

We first introduce the notion of atoms, which was used by Hajek . In the 
space t m , the convex hull of m + 1 linearly independent points in Z m forms a 
simplex . A simplex defined on the set of points {x°, ■ ■ ■ , x m } of Z™ is called 
an atom (defined in [59] § 3) if and only if for some ordering of the set and 
for some permutation (i , ■ ■ ■ ,i m ) of (0, 1, • • • , m), 

x 1 = x° + g ix 

X — X ~f~ Q%2 

■ (1-6) 

x m = x m-l + 

x° = x m +g l0 

where ft , • • • , 9i m are all the elements of T . 

Next we present a characterization of an atom (see [59]), which is essential 
for the optimization result that we obtain in the following sections. Denote 
by [x\ the largest integer smaller than or equal to x. Then the following 
trivially holds 

/ [x+6\d6 = x. (1.7) 
Jo 

Given z £ M m , 6 <E M, define the vector u z {0) in Z m : 

ui(0) = [e + Zl \-[e\, 

uKP) = [6 + zi + ■ ■ ■ + Zi\ - [6 + zi + ■ ■ ■ + Zi-i\ , i = 2, ■ ■ ■ , m. 
Then by (1.7), 

/ u z (9)d9 = z. (1.8) 
Jo 

u z {6) is periodic in 6 with period 1, and piecewise constant with at most 
m + 1 jumps per period. Thus, the set {u z {6) : < 6 < 1} contains at most 
m + 1 vectors, all integer valued. The next Lemma follows from [59]: 

Lemma 4. A point z is contained in an atom, say S(z), if and only if the 
extreme points of S(z) contain {u z (9) : < < 1}. A point z is in the 
interior of atom S(z) if and only if the extreme points of S(z) equal {u z {6) : 
< 9 < 1}. 
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Note that z may be a point in the intersection of two or more atoms. But 
each point z £ t m is contained in some atom, say S(z), and it can thus be 
expressed as the convex combination given by (1.8) of the extreme points of 
S(z). 

For any function / on Z m , we define the corresponding function / on M m 
as follows. It agrees with / on Z™, and its value on an arbitrary point in 
z £ W 1 is obtained as the corresponding linear interpolation of the values of 
/ on the extreme points of the atom S(z). Note that / is uniquely defined, 
if z belongs to atoms Si and 52 then the points u z {6) belong to the extreme 
points of Si n 52 . 

The following theorem establishes the equivalence between multimodular- 
ity of the discrete function and convexity of its continuous extension. In [59], 
the "only if" part was proved (this is the hard part), while the "if" part was 
omitted. In [27], the equivalence is established in the more general context 
of general multimodular triangulations. However, in both cases, the proofs 
are rather involved and use separating hyperplane techniques. The proof pre- 
sented here is elementary and only uses a discrete counterpart of the second 
derivative argument. Furthermore, this proof technique enables us to prove 
this equivalence on special convex subsets of Z m , as shown in Corollary 10. 
A more general presentation is done in Chapter 12. 

Theorem 1. / is multimodular if and only if f is convex. 

Proof, "only if" : The function / is continuous by definition. Moreover, along 
any direction d, it has only a discrete number of isolated points where it is 
not differentiable. By using the characterization of convexity given in [96], 
we will check convexity at a point z by showing that for point z, and any 
direction d, the right derivative is greater than or equal to the left derivative. 
It obviously suffices to check at points that are on the boundary of an atom, 
since, by definition, / is linear in the interior of atoms. Hence, we first assume 
that the point z is on the interior of a face (of dimension m — 1) which is 
common between two adjacent atoms. Without loss of generality, assume that 
the atoms (defined below by their extreme points) are 

A = A(xq,xi, ■ ■ ■ ,x m ) and A = A(x , x\, ■ ■ ■ , x m ). 

where Xi satisfy (1.6) and 

x\ = x + 9i 2 , x 2 = x\ + . 

Case 1: (see Fig. 1.1) g h = -e x = (-1, 0, • • • ,0), g i2 = e m = (0, 0, ■ ■ ■ , 1). 

Decompose direction d in its projection 3 2 in the common face between 
the two atoms and in the component di along the direction (x\ — Xi). In 
the direction <9 2 , the left and right derivatives are equal. In the direction di, 
the right^ derivative is a constant c, depending on the length of di, times 
f{x\) — f(z). The left derivative is c(f(z) — f(xi)). Omitting the constant c, 
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Fig. 1.1. Checking convexity at an interior point z of the common face, Case 1. 



and using point z = \{xo + x 2 ) hence 2f(z) = f{xo) + f(x2), we get for the 
difference 

= (f(xl)-f( Xo ))-(f( X2 )-f( Xl )) (1.9) 
The fact that (1.9) is nonnegative follows by applying (1.1) with x = xo, and 

x\ — Xq + 6 m 

xi = x - ei + e m 
x\ = x - ei. 

Case 2: = e m and ^ 2 = — ei. It is handled as Case 1. 
Case 3: (see Fig. 1.2) = s 2 = (1, — 1, 0, • • • ,0), <?i 2 = —e\. We set x = xo, 
and x| = xo — e\,X2 = xo— e\+S2, x\ = xo+S2- We decompose d along — e\ on 
its projection <9 2 in the common face, and in the projection d\ along (x| — xq). 
As in Case 1, it suffices to consider the direction d\. The right derivative in 
this direction is f(x\) — f(xo), and the left derivative is /(x 2 ) — f(x\) (both 
up to a multiplicative constant). 

The difference between the right and left derivatives is indeed nonnegative: 
f{x\) — f{xo) — (f(x2) — f(xi)) > 0. This is obtained again by applying (1.1). 
Case 4: g ix = s 2 = (1, —1, 0, • • • ,0), gi 2 = S3. In this case, we project d along 
(x| — xi), and the analysis is as for Case 1. 

All other cases, in which z is in the interior of a face (of dimension m — 1), 
common to two adjacent atoms, are similar to one of those considered above. 
It now remains to consider the case where the direction d in point z crosses 
from atom A to atom A, and A n A is of dimension at most m — 2. In that 
case, we consider a cylinder C in direction d containing point z and with 
an arbitrarily small diameter. This cylinder intersects atoms A and A and 
is covered by atoms. We consider the projection Pc of C along direction d. 
This has dimension m—1. The intersection (say of dimension k) of C with an 
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Projection of the common face 
of dimension m — 1 




Fig. 1.2. Checking convexity at a point on the common face, Case 3 



atom is projected on the intersection of the projections and has dimension at 
most k. Therefore, Pc is almost everywhere (in Lebesgue measure) covered 
with projections of dimension m — 1. We can find a line L that belongs to C 
and intersects A and A, with direction d, and which projection is a point in 
Pc not belonging to intersections of dimension smaller than m—1. Therefore, 
we can claim that L only intersects faces of atoms of dimension m—1. The 
convexity in point z and direction d now follows from the convexity in points 
Zi corresponding to the intersections of line L with all the intermediate atoms 
between A and A and a continuity argument. 
"if" : Consider an arbitrary point Xo and any two distinct elements gi,gj in 
T. We have to show that 

f(x ) + f(x 2 ) - fix,) - f{x\) < 0, (1.10) 

i def „ a a . def . , 

where x\ = xo + gi, x\ = xo + gj, x 2 = x\ + gj = x\ + gi, where = is used 

to define the new variable x\ to be equal to xo + gi. 

Define z = \{x\ + x\) = \{x$ + x^) and consider the line segment x\ — > 
z — ► x\. The left derivative (l.d.) and right derivative (r.d.) in z are given by 

l-d- = f{\{xi+xl)) - /(xi) = l -f{x Q ) + \f(x 2 ) - /(xi), 

T.d. = ]{x\) - J{ l -{X + X 2 )) = f{x\) - \f{ Xo ) - \f{ X2 ). 

Since / is convex, r.d. — l.d. is non-negative, and hence (1.10) holds. q 

We can restrict the notion of multimodularity to some convex sets of Z ra . 
Let A be a convex set which is a union of a set of atoms (or of faces of 
atoms). We restrict the definition of multimodularity to directions that only 
lead to points in A. More precisely, we say that / is multimodular in A if the 
following holds. 

If xq, xq + gi, xq + gj, xq + gi + gj are all elements of A then 



/(x ) + /(x + gi + gj) - f(x + gi) - f(x + gj) < 0. 
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Corollary 1. The function f is multimodular in A if and only if f is convex 
on A. 

Proof. It should be clear from the proof of Theorem 1 that the equivalence 
of the multimodularity of / and the convexity of / still holds if we restrict 
the function / to A. □ 

The restriction to A = N™ turns out to be essential for the application of 
Theorem 1 presented in Chapter 4. Indeed, the function considered is only 
defined on non-negative coordinates. Also, in Section 1.5.1, an even smaller 
set A is used in order to consider a case of constrained optimization (see 
Lemma 9). 

A second corollary of Theorem 1 concerns the minimization of multimod- 
ular functions. For a function defined on A, we call x a local minimum on A 
if fix) < f{x ± e;) for all i such that x ± e; is in A. 

Corollary 2. Let the function f be multimodular in A. Then a local mini- 
mum is a global minimum on A. 

Proof. If / is multimodular in A, then / is jxmvex in A, and is linear on the 
(faces of) atoms forming A. The graph of / (i.e. {x : 3y s.t. x > f{y)}) is a 
convex polytope. Therefore, all the local minima are global minima and are 
extreme points of atoms. rj 

Next we consider the integer convexity properties of a function /. A func- 
tion / is said to be integer convex if the following holds. For vectors x and d 
in Z™, we have 

f( x + d)-f(x)>f(x)-f(x-d). 
Theorem 2. Let f be multimodular. Then it is integer convex. 

Proof. Define d£(x):= the right derivative of / at x in the direction h and 
d fl (x):= the left derivative of / at x in the direction h. Since / is convex 
(Theorem 1) then 

d+(x)>d h (x). (1.11) 
Since f(y) = f(y) at the integer points, and since / is convex, we have 

g +(x] < fix + ft) - f_M g- (x) > fix) ~ f{x - ft) 

where |ft| is the L 2 norm of ft. This, together with (1.11) imply the integer 
convexity of /. rj 

The converse of the above theorem is not true: 
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Counter-example 3 Consider the convex function / : N m — ► M given by 
f(x) = maxi = i r . j7n Xi. It is integer convex since it is the maximum of convex 
(linear) functions. However, it is not multimodular. Indeed, consider m = 2, 
x = (i + 1, i) for some integer i. Then 

2i + 2 = f{x - ei + e 2 ) + /(x) > /(x - ei) + /(x + e 2 ) = 2i + 1. 

Hence / is not multimodular. 

For integer convex functions we need not have the useful property that a 
local minimum is a global minimum. This is illustrated in the next counter- 
example. 

Counter-example 4 Consider the space ({0,1, 2}) 2 . Define the function g 
such that 

5 (0,2) = -1, 5 (1,2) =2, 5(2,2) =5, 
5(0,1)= 2, 5(1,1) = 1, ff(2,l)=0, 
5(0,0)= 5, 5(1,0) =4, 5(2,0) =3. 

One can easily check that g is an integer convex function, but not multimod- 
ular since 5((1, 2)+b ) +g((l, 2) + b 2 ) = < 4 = g((l, 2)) +g((l, 2)+b + b 2 ). 
Starting the local search algorithm at coordinate (2,1) shows that all neigh- 
bors have values, which are greater than 0. However, the global minimum is 
fl((0,2))=-l. 

This shows that when dealing with discrete functions, the counterpart of 
convexity is not integer convexity but rather multimodularity, which insures, 
for example, that a local minimum is a global minimum point (see Corollary 
2). 

The following result will be very useful when cost functions involve arbi- 
trary convex functions of the quantities of interest, especially to prove opti- 
mality in the convex increasing order. 

Theorem 5. The following statements are equivalent, 

(i) f is multimodular and for all v,w £ T, max(/(x + v),f(x + w)) > 
max(/(x), f(x +v+ w)). 

(ii) For all h : M — > M convex increasing, h(f) is multimodular. 

Proof. We will use the notation introduced in [88]. Condition (i) is equivalent 
to the fact that the two-dimensional vector (f(x),f(x + v + w)) is weakly 
submajorized by (f(x + v), f(x + w)), denoted (f(x),f(x + v + w)) ~< w (f(x + 
v),f(x +w)). By using proposition 4.C.l.b in [88] then this implies that for 
any convex increasing function h, h(f(x)) + h(f(x + v + w)) < h((f(x + v)) + 
h(f(x + w)). This is exactly the multimodularity of the function h(f). 

As for the converse, note that (ii) implies in particular that h(f) is multi- 
modular for all h continuous, convex and increasing. Using Proposition 4.B.2 
in [88], this implies that (f(x),f(x + v + w)) < w (f(x + v),f(x + w)). By 
definition of weak majorization, that is the same as statement (i). □ 
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1.3 The optimality of bracket policies for a single 
criterion 

In this section, we will present a rather general framework under which mul- 
timodularity can be used in order to optimize a cost function based on a 
sequence of functions which will represent a quantity of interest in a given 
model. The problems that can be solved using this framework include the 
minimization of the average workload in a queue under general stationary 
assumptions, as well as many other similar problems (see for example Chap- 
ter 4). § 1.5.1 that presents a precise instance of such a problem. 

The sequence of functions considered can be interpreted as cost functions, 
which are defined on a common sequence (a\ , a-i , ■ ■ ■ ) of integers which we 
call a control sequence. For example, a; can be the number of admitted jobs 
at the ith arriving epoch for an admission control problem in a queue. The 
control sequences will all belong to a set A which is a convex union of atoms 
in Z k . Our objective is to study optimization properties of Cesaro averages 
of the cost functions over the class of control sequences. 

Consider a sequence of functions fk ■ — ► R+ U {oo} that satisfy the 
following assumptions: 

— < 1 > fk is multimodular on A. 

- < 2 > / fc (oi,--- ,o fc ) > fk-i(a 2 ,--- ,a>k), Vfc > 1; 

For a given sequence {ofc}, we define the cost g(a) as 



Definition 2. Let p and 9 be two positive reals. We define the bracket se- 
quence {of (0)}fceN with rate p and initial phase 6 as, 



where [x\ denotes the largest integer no larger than x. 

Note that when p and k are fixed, the set {a v k (0),0 < 8 < 1} are extreme 
points of an atom containing the point (p, p, ■ ■ ■ ,p). 

The aim of this section is to prove that this sequence minimizes the func- 
tion g, provided that some conditions (including < 1 > and < 2 >) above 
hold. The sequence (1.13) was used by Hajek in [59], and we use several 
properties of the bracket sequence established in [59] . To establish the main 
optimization results, we need the following technical Lemma. 

Lemma 5. If fk satisfies assumptions < 1 >,< 2 >, then the function fk 
satisfies assumption < 2 > for positive real numbers. 




(1.12) 



a' 



p k (6) = [kp + 9\ - [(k-l)p + 0\ 



(1.13) 



The proof of this lemma is given in Appendix (section 1.8). 
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Lemma 6. Under assumptions < 1 > and < 2 >, let O be a random vari- 
able, uniformly distributed in [0,1), and denote the expectation w.r.t. by 
Eq. Then 

Jim E f N {a{{&),--- ,<(©)) = lim J N (p,p,--- ,p). (1.14) 

Proof. We have for all TV, 

£ /jv«(6>),--- ,<(©)) =f N (p,--- ,P). (1-15) 

(This follows (1.8), from Lemma 4, and the fact that Jn is affine on each atom, 
and agrees with /jv for the extreme points of the atom.) Since Jn{p,P, ■■ ■ ,p) 
is increasing in N by Lemma 5, the limit in N exists (it is possibly infinite). 

□ 

Definition 3. We call the sequence {a p (&)} the randomized bracket policy 
with rate p. 

1.3.1 Upper Bounds 

Lemma 7. Under assumptions < 1 > and < 2 >, for every 6 £ [0, 1), 
1 N 

J im ^E^KW'--- XW) < lim Mp,JV ,J»). (1-16) 

iv — »-oo iV Jv— »-oo 

7L=1 

Proof. Define 

/ m («,?)=/ m (aW,-,<W). 
/ m is periodic (in 6) with period 1. Define 

fL(0,P) = Ua p _ m+1 (6),---,a p (6)). 

Then we have 

fL(0',p)=fm(0,p) where 0'=0+mp, (1.17) 

Indeed, 

/;^',p)=/™(a p m+1 (e'),---,o^')) 

= fm(a p _ m+1 (6 +mp),--- ,og(6»+mp)) = f m (0,p), 

where the last equality follows from the fact that a v _ m+k {6 + mp) = a v k {6), 
k = 1, ■ ■ ■ , m. f' m is again periodic w.r.t. 6, with period 1, and is increasing 
in m so that the following limit exists (possibly infinite): 
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fL(o, P )= iim f m [e, P ). 

Moreover, we have that EQf' m {Q,p) = f m (p, ■ ■ ■ ,p), where & be a random 
variable, uniformly distributed in [0, 1] (this follows from (1.8), from Lemma 
4, and fact that f n is affine on each atom, and agrees with f n for the extreme 
points of the atom). Hence, 

EefL(@,P) = Jim f N (p,P, ■ ■ ■ ,p). (1.18) 

Consider now the bracket sequence for fixed 9. Then 

1 N 

^£/™K(0),---,<(0)) 

m=l 

i N 

<^E M a - N +m+ M ■ ■ ■ > asw. ■ ■ ■ . 

m=l 

i w 

m=l 

The last inequality follows from assumption < 2 > for the functions fk, as 
well as an argument similar to the one used in (1.17). 

If p is irrational, applying the ergodic theorem of Weyl and Von Neumann 
([102]), we have 

1 N 

Krn^ - £ fU-mp + e,p)= E e fL(0,P)- 

From Equation (1.18), we have Eef'^O^) = limjv^oo Jn(p,P, ■ ■ ■ ,p)- 
This implies that if p is irrational, 

1 N 

J im m £ fU-mp + e,p) = lim f N (p,p,--- ,p). (1.19) 

iv — >oo iV iv— >oo 

If p is rational, then p = q/d where and d are relatively prime and 
d > 1. This implies that the sequence (o P _(jv_ m )(^)i ' ' ' > a o(^)' ' " > °m(^)) ^ s 
constant if #( mod 1) £ [j/d, (j + l)/<2), f° r an J- Therefore, f' m {0,p) is also 
constant on these intervals and by passage to the limit, f^{0,p) is constant 
on these intervals. Now, note that Frac(# — mp) £ [j/d, (j + l)/d) for exactly 
one value of m out of d consecutive values of m because q are d are relatively 
prime. Now, we have 

N d-l 

lirn^ - £ fU-mp + 8,p) = - £ fL(m/d,p) = Eef'^P)- 

m=l m=l 

Equation (1.18) concludes this case as well. □ 



24 1 Multimodularity, Convexity and Optimization 
1.3.2 Lower Bounds 

In this subsection, we establish lower bounds for the discounted cost for all 
control sequences {ofc}. This then serves for obtaining a lower bound on the 
average cost. Here, we use the following assumption for the functions 

— < 3 > For any sequence {cik} 3 a sequence {bk} such that 

Vfc,m with k > m, fk(h, ■ ■ ■ ,b k - m ,ai, ■ ■ ■ ,a m ) = / m (oi, • • • ,a m ). 

We use the notions defined in the previous sections. 

Let us fix the sequence {ofc}, as well as some arbitrary integer, N. We 

define p a = (1 - a) J2T=i ak ~ 1(l k- 

Now, using assumptions < 1 > through < 3 >, we have 



^(1 - a)a n ~ 1 f„(ai,a 2 ■■■ ,a n ) 

71=1 

JV 

> ^(1 - a)a n ~ l f N {bi, ■ ■ ■ ,b N -n,ai,(i2, - ■ ■ ,a-n 



71=1 



+ ^ - a ) an ~ 1 fN(a n -N+i,a n -N+2,--- ,a„) 

n=N+l 

N 

= 5^(1 - a)a n_1 75v(6i, ■ ■ ■ ,b N -„,ai,a 2 ,- ■■ >%) 



71=1 



+ ^2 ( 1 ~ a ) an l fN{dn-N+l,an-N+2,---,an) 
n=N+l 

N 

> /jv( ^(1 - a)a n_1 (6i, • • • ,6w-7i,oi,02, ■ ■ ■ ,a„) 



+ ^ (l-a)a n_1 (a n _jv+i,a n _jv+2,--- ,On)) (1-20) 

7l=JV + l 

/ JV-2 iV-3 \ 

= In [h Y, (! - + 62 a ) a ™ + """V., ■ --Pel)- 21 ) 



71=0 71=0 



Equation (1.20) follows from Jensen's inequality, since by Theorem 1, the 
function /jv is convex, and since the coefficients (1 — a)a n_1 are nonnegative 
and sum to 1. Define 



JV-2 JV-3 



B(N, a,p) = f N (h (1 - a)a n + a N p, b 2 ^ (1 - + a JV_1 P, ■ ■ ■ , J>)- 



71=0 71=0 



Note that -B is defined for a fixed sequence {ofc}. Also note that B(N,a,p) 
is lower semi-continuous in a and in p. 
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Using Lemma 10 given in Appendix (section 1.9), we derive the following 
lower bounds 

Lemma 8. Under assumptions < 1 >, < 2 > and < 3 >, 

^ m 

lim — V /„(ai, ■■■ ,a n )> inf f N (q, ■■■ ,q), 

n=l 

where C is the set of all limit points of p a as a | 1. 
Proof. 

^ m oo 

lim — V" /„(ai, • • • ,a n ) > lim(l - a) V] a n_1 /n(ai, • ■ ■ ,a n ) 

m— »oo m afl '—^ 

71=1 71 — 1 

> lim B(N,a, p a ) 

a]l 

>mi qeC B{N,l,q), (1.22) 

The Lemma follows since for any given p, by definition of B, B(N, l,p) = 
7n(p,P,--- ,P)- □ 

1.3.3 Optimality of the Bracket Sequences 

Theorem 6. Under assumptions < 1 > ,< 2 > and < 3 >, and given some 
p e [0,1], if the functions /*(ai,-- - , a*,) are increasing in all a>i, then the 
bracket sequence a p (6) for any 8 £ [0, 1), minimizes the average cost g(a) 
over all sequences that satisfy the constraint: 

1 N 

Mm T7 a n > p. 

Proof. We denote by 

N 



P = Mm ^5Z a 

71=1 



By using Lemma 10 in the Appendix (section 1.9), 

p < p < limp a = ini{q, q € £}. 

If the functions {/&} are increasing, then B is increasing in p, therefore, 

g(a) >mi qeC B(N,l,q) > B(N,l,p) = f N (p, ■ ■ ■ ,p), (1.23) 
by Lemma 8. If we let N go to infinity, we get 
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1 N _ 

g(a) = Jim — V /„(ai, • • • , a„) > Jim /jv(iV -,?>)■ (1-24) 

71=1 

Lemma 7 shows that linijv^oo Jn(p,P, -;P) > g(a p (0)). Thus g(a) > g(a p (6)). 
□ 

When the functions are decreasing, we have the analogous result. 

Theorem 7. Under assumptions < 1 > ; < 2 > and < 3 >, and given some 
p £ [0, 1], and any 6 £ [0, 1], if the functions fk(a>i, ■ ■ ■ ,a>k) are decreasing in 
all a>i, then the bracket sequence a p (6) minimizes the average cost g(a) over 
all sequences that satisfy the constraint: 



! N 

lim — > a n < p. 



71=1 

Proof. The proof is similar to the previous one, using the fact that if 

JV 



A 1 N 

p= lim -^y^a n 



=1 
then 

p>p> limp a = supj^, q £ £}. 



□ 



1.4 The optimality of bracket policies for multiple 
criteria 

In this section, we establish general conditions under which the bracket policy 
is optimal when the cost function depends on multiple criteria. While the 
single criterion framework has application in admission control, this multiple 
criteria approach has applications for routing control to several queues. For 
instance, for the routing to several identical -/Gl/1 queues, it is known that 
the round robin routing is optimal in separable-convex increasing order [82] . 
In order to obtain this type of results in our framework, we shall show the 
existence of optimal asymptotic fractions in a very general case. 

From now on, we study the following general optimization problem. Con- 
sider K sequences of functions f^,i = 1, • • • , K. Each sequence of functions 
f l n will only depend on the sequence of the ith coordinates a 1 in a, and will 
satisfy assumptions < 1 >,< 2 > and < 3 >, as in Section 1.3. 

A policy is a sequence a = (ai, a2, ■ ■ ■ ), where a n is a vector taking values 
in {0, 1} K . We consider the additional constraint that for every integer j, 
only one of the components of Oj may be different from 0. A policy satisfying 
this constraint is called feasible . 
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Let ft be a convex increasing function from M. K to M. Define 

1 N 

g(a) ^ lim ^^^(a 1 ),-,/^)). (1.25) 

iv — >oo ./V r ' 

Following notations introduced in Section 1.3, we get a bounding function 
called Bi(N,a,p) for coordinate i. Here, we denote by 

Bi(a,p) = sup Bi(N, a, p), 

N 

and 

Bi(p) = sup Bi(a,p). 

a<l 

Note that by convexity of f$ and Lemma 5, Bi(a,p) and Bi(p) are 
continuous from below in (a,p) and p, respectively. 

Our objective is to minimize g(a), with no constraints on the asymptotic 
fractions. 

Theorem 8. Assume that for all i, the functions f„ satisfy assumptions 
< 1 >,< 2 > and < 3 >. The following lower bound holds for all policies: 

g(a)> inf ft(Bi(pi), ■ ■ ■ , B K {p K )). 

PlH hPK=l 

Proof. Due to Lemma 10 in the Appendix C (section 1.9), Jensen's inequality 
and Equation (1.21), we have 



J™ w-.- "./^ 

71=1 

OO 

> ^(I-qJ^q"- 1 ^,--- ,/, 

a— *-! ' ^ 



> hm ft (1 - a) a" -1 /;, ■ ■ ■ , (1 - a) £ 

> nEA(fli(a,pf(a)),-- - ,B*(a,jfc(a))), (1.26) 



where 



p>)^(l-a)f> fc -V fc . (1.27) 



fc=i 



We note that YliLi Pti a ) = 1- Hence, one may choose a sequence a„ | 1 such 
that the following limits exist: 

lim pf(a n ) =Pi, i = l,--,K (1.28) 
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and J2i=i Pi = 1- From the lower semi-continuity of Bi(a,pi) in pi and a we 
get from (1.26) 

g(a) > pi), • • • , , B K (l,p K )) (1.29) 

> inf /^(l,^),--- ,B K (l,p K )). 

PiH hPK=l 

□ 

Note that there exists some p* that achieves the infimum 

inf /^(l,^),--- ,B K {l,p K )), (1.30) 

PiH hPK = l 

for • • • , Sx(l, Pk)) is continuous from below in p = (pi, ■ ■ ■ ,Pk)- 

Balanced policies 

Consider the bracket policy a v (6) given by 

4M = [k P * + 6i\ - [(k - l)p* + Bi\ . (1.31) 

There are some p* for which the condition of feasibility of the policy a p * (6) 
is satisfied, that is, there exists some 6 = (6\, ■ ■ ■ ,6k), such that the policy 
a p (8) given in (1.31) is feasible. These p* are called balanceable and we shall 
come back to them in more details in Sections 4.6.1 and 2.2. 

Theorem 9. Assume that for all i, the functions satisfy assumptions 
< 1 > ; < 2 > and < 3 >. Assume that h is linear increasing and that p* 
is balanceable. Then a v (6) is optimal for the average cost, i.e. it minimizes 
g(a) over all feasible policies. 

Proof. The proof follows directly from Lemma 7 together with Theorem 8. rj 

The balance condition on p* is still not completely characterized, however, 
we can mention two simple cases for which p* is balanceable. i.e. for which 
there exist some 6 = {6\, ■ ■ ■ ,6k), such that a v (6) is feasible. 

- CI: K = 2. 

— C2: K criteria with symmetric costs, i.e. h(x) = an d ah f % (as 
functions of a 1 ) are equal. 

Corollary 3. (i) Consider the case CI. For p* that achieves the minimum 
in (1.30), the bracket policy for rate p* and some initial phase 8 is feasible 
and optimal. 

(ii) Consider the case C2. By symmetry, the bracket policy with p = 1/K for 
some initial phase 6 is feasible and optimal. 

Remark 2. Hajek gives an argument ([59] Remark (5) p. 554) that shows for 
K = 2 the optimality of the bracket policy with rate vector (p, 1 — p) in 
the restricted class of policies a such that linijv^oo ^ Yl n =i a n = P- Note 
that Corollary 3 (ii) shows the optimality of a bracket policy for all control 
sequences a. 
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Next, we restrict again to the case of a single objective (K = 1), and show 
that the results of the previous section can be extended. More precisely, we 
show that a bracket policy is optimal in a stronger sense. 

Corollary 4. Under the conditions of Theorem 6, given some p £ [0, 1], and 
any 6 £ [0,1], the bracket policy a p (6) minimizes the average cost g(a) over 
all policies that satisfy the constraint: 

lim pi(a) > p\. (1.32) 

a— >1 

where Pi (a) is defined in (1.27). 

Note that the constraint lim M >OQ J2n=i ttn — P 1 ( m Theorem 6) im- 
plies (1.32), due to Lemma 10 in the Appendix (section 1.9). Therefore the 
minimization in Theorem 6 is over a subclass of the set of policies on which 
minimization is performed in Corollary 4. Thus, Corollary 4 implies that a 
policy a that satisfies (1.32) does not perform better than the bracket policy 

(with p = pi) even if limjv^oo w £n=i a n < Pi- 
Proof of Corollary 4: 

Proof. Choose an arbitrary policy a that satisfies (1.32). Choose a subse- 
quence q„ | 1 such that lim^oo pf(a n ) > p\. The proof now follows by 
combining Lemma 7 with (1.32). j-j 

1.5 Application of the optimization theorems 

In this section, we will briefly present some optimization problems that fit 
the framework presented in the previous sections. Other models that require 
a more extensive analysis can be found in future chapters. 

Remark 3. It seems difficult to give a consistent meaning to the workload 
associated with a negative number of customers. One may even doubt that 
there is a satisfying way to do so that will also preserve multimodularity. This 
is why we have not considered the quantity E^t (ai , • • • , a*,) on Z fc , but 
only on N fc , which is a convex union of atoms. Therefore, all the optimization 
framework constructed in section 1.3 can be used in this case. 

1.5.1 Applications in high-speed telecommunication systems 

We consider a simple model composed of a controlled D/D/l queue with 
service times a n = a and inter-arrival times r n = r all deterministic. As- 
sume that the available actions are (corresponding to rejecting an arriving 
customer) and 1 (corresponding to acceptance of an arriving customer) . 

The type of problem we consider is typical in high speed telecommu- 
nications networks, and in particular, to the ATM (Asynchronous Transfer 
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Mode) . The latter has been chosen by the standardization committee ITU-T 1 
[77] as the main standard for integration of services in broadband networks. 
In order to handle efficiently a large variety of applications, such as voice, 
data, video and file transfer, cells of fixed size are used, giving rise to our 
model that uses fixed service times. Fixed inter- arrival times are typical for 
isochronous applications (voice, video) and also for large file transfer. 

Two important measures of quality of services in ATM networks are loss 
probabilities (CLR - Cell Loss Ratios) and delays. According to the ATM 
standard [77], when a CBR (Constant Bit Rate) session is established, the 
network should provide a guarantee that these two measures are bounded by 
given constants. Since the available sources are limited and, moreover, might 
be shared with other applications, a typical objective of the network is to 
minimize the delay of the CBR session while meeting the constraint on the 
loss probabilities. Losses might be due either to overflow, or to deliberate 
packet discarding by the network (e.g. to allow the resources to be available 
for other applications). The problem can be formulated in our framework 
as one of discarding cells so as to minimize the average queue size (i.e. the 
workload in the system) which is known to be proportional to the average 
sojourn time (due to Little's law), subject to a lower bound p on the average 
cell discarding rate. 

We now describe the state evolution of the system. If x n denotes the 
amount of workload in the system immediately after the nth arrival that 
occurs after time 0, and the system is initially empty (at time 0), then 

x n = max(x„_i — r, 0) + a n a. 

The solution of this recursion is given by the expansion of the Lindley's 
equation: 

f ™ 1 I 

x n = /n(ai, ■ • • ,a n ) = max < 0,^(a fc <r - r), j = 1, ■ ■ ■ , n - 1 > + a n a. 

{ k=j J 

(1.33) 

We show by a simple inductive argument that for all n, x n satisfies: 

x„(a + v\) + x n (a + v 2 ) > x n (a) + x n (a + v 2 + vi), (1-34) 
x n (a + vi) V x n (a + v 2 ) > x n (a) V x n (a + v 2 + v\). (1.35) 

The function x\{a) = a\a clearly satisfies (1.34) and (1.35). 
if vi, V2 are in T\{e n , s n }, then by induction 

1 The ITU-T is a group of the standardization organization ITU (International 
Telecommunications Union), and is accessible through their WEB address: 
http: / /www.itu.int /ITU-T/. 
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x n (a + vi) + x n (a + v 2 ) 

= {x n -i{a + di)-tV0) + a n a + (x n -i(a + h 2 )-tV0) + a n a 
= {x n -i{a + vi) + x n _i(a + v 2 ) - 2r 

Vx n -i(a + vi) - t V x n -i(a + h 2 )-tV0) + 2a n a 

> {xn-i (a) + x n -i (a + v 2 + vi ) - 2r 

Vx n -i (a) - r V x n -i (a + D 2 l»i)-TV0)l 2a n a 
= x n (a) + x n (a + v 2 + vi). 

x n (a + vi) V x n (a + v 2 ) = (x n -i{a + «i)-tV x n -i{a + » 2 )-tV0) + a n a 
= ((x n -i(a + vi) V x n -i(a + v 2 )) - r V 0) + a n a 
> x n (a) V x n (a + v 2 + v\). 

If v\ = s n and v 2 is in ^F\{e n ,s n }, then Equation (1.34) is obtained 
similarly, by induction. As for Equation (1.35), the proof is slightly different. 

x n (a + vi) V x n (a + v 2 ) 

= (x n -i(a + e„_i) - t - a V -a V a: n _i(a + d 2 )-tV0) + a K a 
= (x n -i(a) - r V a: n _i(a + h 2 )-tV0) + o n a 

> a;„(a) V x n (a + v 2 + v\). 

If Hi = e n , x n (a + e n ) +x n (a + v 2 ) = x n (a) +x n (a + v 2 ) +a = x n (a)+x n (a + 
v 2 + e„), and 

£n(a + e„) V x n (a + v 2 ) = x n (a) + a 

> x n (a) V x n (a + v 2 ) + a 
= x n (a) V x n (a + v 2 + Hi). 

Our goal is now to obtain a policy a* that minimizes an expected average 
cost related to the amount of work in the system at arrival epochs. The cost 
to be minimized is thus 

1 N 

9 ^ = n [ SL N S f^ 0,1 ' ' ' ' ,an ^ 

n=l 

subject to the constraint: 

1 N 

lim — Va n > p*. 

Consider first the case of a queue with infinite capacity. Then, it follows from 
Theorem 6 that a bracket policy with rate p* and arbitrary 6 is optimal. The 
assumptions of the Theorem indeed hold: 
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— f n (in (1.33)) is indeed monotone increasing in af, 

— Property < 3 > (in Subsection 1.3.2) holds by choosing b k = 0, since 

fk(ai, ■■■ ,a k ) = f m ( 0, ■ • • ,0 ,ffli, ■ • • ,a k ), k <m; (1.36) 

m — k 

— By combining (1.36) with the first monotonicity property, we get 

fk-l((l2, • • • , Ofc) = /fc(0, 0,2, ■ ■ ■ , Ofc) < /fc(oi, • • • , dfc), 

which establishes Property < 2 > (in the beginning of Section 1.3). 

Consider now a queue with a finite storage capacity for the workload, i.e. 
the workload at the queue at each time instant is bounded by C. When the 
queue is full, the overflow workload is lost. The bracket policy need not be 
optimal anymore, as the following example shows. 

Counter-example 10 (Non optimality of a bracket policy) 
Let r = 1, a = 100, C = 100, p* = 0.01. Assume that the cost to be minimized 
is the average queue length. The bracket policy with rate 0.01 achieves an 
average queue length of 50.5 for any 6. Consider now the periodic policy 
of period 200 that accepts 2 consecutive customers and rejects all following 
ones. After the second acceptance, the amount of work in the system is 100 
due to the limit on the queue capacity, and there is loss of workload (of 99 
units). The average queue length is 25.75. Thus the new policy achieves half 
the queue length as the previous one. 

Although the bracket policy in the above counter example results in a 
larger queue, it has the advantage over the other policy of not creating losses. 
As we now show, a bracket policy is optimal if we restrict to policies with 
the additional constraint that no losses are allowed. Thus, consider the class 
of policies that satisfy the constraint: 

(ai, ■ ■ ■ ,a n ) < C 

where x n is given by (1.33). 

Lemma 9. The set A = {(ai, • • • , a n ) £ K 1 : x n (a\ • • • , a n ) < C} defines a 
convex union of atoms ofZ n . 

Proof. By definition of x n , the set A can also be defined as 

A = {(a u --- ,a n ) :Vj, cij £ N, a 3 > 0, ^a k < J>T }. 

k=j 

(1.37) 

Since A is only formed of integer points, we can also write 
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n 



C + (n- j) T 



A = {(oi,--- 



a 



"n 



) : Vj, cij e N, aj > 0, ^ a fc < |_ 



J}- 



k=j 



(1.38) 



Now, let us consider the constraints one by one. 
— The constraints cij > restrict A to N 71 which is made of a convex union 



projection over the last n — j coordinates, this constraint is a convex union 
of faces of atoms. Therefore, on the whole set, this constraint is a union of 
atoms. 

To finish the proof, remark that the intersection of convex union of atoms 
is a convex union of atoms. □ 

Using now Corollary 1 and Theorem 6, we conclude that a bracket policy 
is again optimal. 

In the above admission control we considered only the possibility of ac- 
cepting or rejecting the whole arriving batch (of 100). In practice, arriving 
batches may correspond to cells originating from different sources, and it is 
often possible to reject only a part of the batch. 

Assume, thus, that the available actions are a £ {0, l, - - , N}, where 
a = i means accepting ?i(100/iV) units of workload. Assume that the batch 
size of 100 is an integer multiple of N. We can thus split an arrival batch and 
accept only a fraction of it; more precisely, we can either reject it, or accept 
1/Nth of the batch, or 2/iVth, etc... The smallest unit of batch which we can 
accept (i.e. N ) is called a mini batch. 

Consider now the bracket policy a*[N] that is given in (1.13) correspond- 
ing to p = p*N. In other words, instead of considering a target fraction p 
of the whole batch to be accepted, which is smaller than (or equal to) 1, 
the new target corresponds to the average number of mini batches to be ac- 
cepted, and can be any real number between and N (in particular, p = N 
will correspond to accepting N mini-batches, i.e. the whole original batch). 

We may repeat the above calculation and show that this policy is optimal 
for the cases of (i) the infinite queue and (ii) the bounded queue, restricting 
to policies that do not generate losses. Moreover, for both cases, this policy 
is better than the one that consists of accepting or rejecting the whole batch 
according to the policy o* defined above, since a* is a feasible policy in our 
new problem, for which a*[iV] is optimal (Theorem 6). 

In order to illustrate the last point, consider N = 10. A bracket policy 
corresponds to acceptance of a mini-batch of 10 units, once every 10 time 
slots. The average queue length obtained by that policy is 5.5, i.e. about ten 
times less than the one obtained when the whole batch was to be accepted 
or rejected. 
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1.6 Clustering versus Smoothing 

We have already seen in the previous section an example where a bracket 
policy is not optimal, and only when adding a further constraint we obtain 
the optimality of a bracket policy. We present here yet another admission 
control example in which a policy that tends to cluster acceptances performs 
better than any bracket policy. 

Consider discrete time; assume that every time unit there is a potential 
arrival (it becomes an actual arrival if it is accepted). Its service time takes 
2.5 units. The fraction of accepted customers is not allowed to be below 1/2. 

We consider two types of regimes of the system: 

(i) The preemptive regime: the packet in service is lost if another one 
arrives while it is served. 

(ii) The nonpreemptive policy: the arriving customer is lost if it finds a 
packet in service. 

The best bracket policy accepts every second arrival. If the preemptive 
regime is used, then under this bracket policy, all accepted packets are lost. If 
the system operates under the nonpreemptive regime, then under this bracket 
policy, every second admitted packet is lost. 

If we use the sequence (110100)°° instead of (101010)°° we get both in 
the preemptive as well as in the nonpreemptive case an improvement: now 
only 1/3 of the admitted packets are lost. Thus a non bracket policy performs 
better than the bracket ones. 

1.7 Appendix A: proof of Lemma 2 

Proof, (a) We shall show that for any w, v £ T, there is a one to one map 
z : z m Z m such that 

A v A w f(x) = f(z(x)) - f(z(x) +v + w)- f(z(x) + v)- f(z(x) + w). (1.39) 

Hence the multimodularity implies A v A w f < 0. Since the map is one to one, 
we get also the converse. In the next four equations we illustrate (1.39) which 
establishes the proof. 

Consider first w = s,, v = sj (v ^ w). Then 



A v A w f(x) 



{Ai-! - AMA^ - Aj)f{x) 



(Ai-! - Ai)(f(x + Bj-i) - f(x + Bj)) 

f{x + ej-i + ei-i) - f{x + ej-i + e*) 



f(x + ej + e;_i) + f(x + ej + e*) 



f( Z + Sj + Si) - f( Z + Sj) - f(z + Si) + f{z) 



(1.40) 
(1.41) 



where z = x + ej + e^. 
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Let v = e m ,w = —e\. Then 

A v A w f{x) = A m {f{x - ei) - /(a;)) 

= fix + e m - ei) - f{x - ei) - /(x + e m ) + fix) (1.42) 

Let v = e m ,w = sj. 

A v A w fix) = AmiAj^ - Aj)fix) 

= Amifix + ej-x) - fix + ej)) 

= fix + ej-i + e m ) - fix + ej-i) - fix + ej + e m ) + fix + ej) 
= fiz + Sj + e m ) - fiz + Sj ) - fiz + e m ) + fiz) (1.43) 

where z = x + ej. 

Let v = —ei,w = sj. 

A v A w fix) = 4„(Aj-i - Aj)fix) 

= ^-eAfi X + e j-l) ~ fi X + e j)) 

= fix + ej-i - ei) - fix + ej-i) - fix + ej - ei) - fix+ej) 
= fiz + s 3 - ei )+ fiz + 8j ) + fiz - ei) + fiz) (1.44) 

where z = x + ej. 

(b.i) For any w € J 7 , 

A ei A w fix) = A w A e Jix) = -A w A_ e Jiz) 

where z = x + e\. The result follows from Lemma 2 (a), 
(b.ii) Without loss of generality, assume that i < j. Then 

AiAjf = I A ei - A ak j j jr A SI + A em J / 

\ k=2 J \l=j+l J 

The proof of (b.ii) is established by applying Lemma 2 (a) and Lemma 2 
(b.i). 

For i < j we have 

iAj - Ai)Ajf = ( - E A ») ( E A « + A ^ I / 

V k=i+l I \I=j+l / 

and (b.iii) is established by applying Lemma 2 (a). For i > j we have 

m — l j 

A 3 =Ai- E A s k - A m + A ei - E^fc- 

k=i+l k=2 

Hence 
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A j (A j -A i )f= U B1 -2>'* - E ^ k -A m + A ei -Y J ^\f. 

V fc=2 / V k=i+l k=2 / 

Again, (b.iii) is established by applying Lemma 2 (a) and 2 (b.i). 
By taking i = j — 1 in Lemma 2 (b.iii), we obtain (b.iv). 
(b.v) For j < i, 

^e,A S] f=^pA Sk +A em ^A Sj f. 

For j > i, 

Ae,A S] f= (a si -J2A Sk ^j A S] f. 

For both cases, the proof is established by applying Lemma 2 (a), and in the 

second case we use also Lemma 2 (b.i). 

(b.vi) 

A sl A Si f = (A em -A ei )A Si f 

The proof is established by applying Lemma 2 (a) and 2 (b.i). 
(b.vii) 

(1.45) 

For i ^ 1, the proof is established by applying Lemma 2 (a) and 2 (b.i), 
where we replace in the summation A S1 by A Sm — A ei . 
For i = 1 it follows from part (b.vi) of this Lemma and setting i = 1 in (1.45). 
(c) A_ ei A e2 < due to (1.4); A_ ei A S2 = A_ ei (A ei - A e J < 0, and 
A e2 A S2 = A e2 (A ei — A e2 ) < due to (1.5). Hence / is multimodular by 
Lemma 2 (a). □ 




1.8 Appendix B: Proof of Lemma 5 

Proof. Let z = (z\,--- ,Zk) € M+. This point belongs to an atom, say 
S(z), made by the extreme points . The numbering of the 

extreme points of the atom is chosen such that according to the base 
T h = (— ei,S2,--- i s fci e fc)i x 1 = x ° — e i- The other indices are arbitrary. 
This implies that x° = x] for all j > 2. If we call P the projection of 

onto l^T 1 along the first coordinate, 

P{s k 3 )=P{s)- 1 )ii2<j<k 
P(s k 2 ) = -et 1 
P(e$) = 
P(et) = et 1 
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These equalities imply that P(x°) = P(x 1 ) and P(x 1 ),--- ,P(x k ) form 
an atom in R^T 1 , as follows from the definition of an atom, and P(z) belongs 
to this atom. Let 

(zi,z 2 ,--- ,Zk) = ^1 - x ° + a i xl H \-ot k x k , 

then 

(z 2 ,--- ,z k ) = P{z 1 ,z 2 ,--- ,z k ) 

= (l-Y^a^J P(x°) + aiP^ 1 ) + • • • + a k P{x k ) 

= (l~J2 a ^J P(x 1 ) + ---+a k P(x k ). 



Now, 



fk(zi,z 2 , ■■■ ,z k ) = ^1 - XJ a *j fk{x°) + aufkix 1 ) H \-a k fk{x k ) 

> ^1-E Q ^ fk-i(P(x°)) + ---+a k f k ^(P(x k )) 
= ^1-E Q ^ fk-i(P(x 1 )) + ---+a k f k ^ 1 (P(x k )) 

= fk-l(z 2 ,--- ,Z k ). 

□ 

1.9 Appendix C: average and weighted costs 

The following Lemma is often used in applications of optimal control (or 
games) with an average cost criteria (see e.g. [105]), yet it is not easy to find 
its proof in the literature in the format in which we want to apply it. 

Lemma 10. Consider a sequence a n of real numbers all having the same 
sign. Then, 

N oo 

lim — y^a n > Iim(l -a) Va^'fli (1.46) 

n— >oo JV ^ — ' a— >1 ^ — ' 

n=l k=l 

oo JV 

> lim(l-a)^a* _1 a* > lim -r;^an=P ( L47 ) 

k=l n ^°° iV n=l 
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Proof. Note that 

k 



fc-i 



fc=i 



fc=l \z=i 



1 oo 



(l-a) 



fc=i 



Hence 



For any e > 0, choose N t such that 

1 W 



71=1 



(1.48) 
(1.49) 



oo oo / ^ k \ 

(1 - a) £ a fc - V - p = (1 - a) 2 £ - £ a, - p) fca^ 1 . (1.50) 

fc=i fc=i V 1=1 ) 



for all N > N e . Then the right-hand side of (1.50) is bounded below by 

(JV e -l / k \ oo \ 

E U^-'Efa**- 1 
fc=i V 1=1 / fc=« e / 

- hp ] - e(l - a)~ 



>(l-aY 



N e max 

Kk<N c 



> -2e 



for a sufficiently close to 1. This establishes (1.47), and (1.46) is obtained 
similarly. q 
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2.1 Introduction 

In this chapter, we will introduce the notion of balanced sequences, which 
is closely related to the notion of Sturmian sequences [85] as well as exactly 
covering sequences. 

Those sequences are binary sequences where the ones are distributed "as 
evenly as possible" while satisfying a density constraint. 

This presentation is not exhaustive and many other related articles and 
books can be consulted for further investigation on this topic [113, 75, 16, 
28, 107, 46, 84]. Although the chapter is self-contained and presents several 
results which are of interest by their own, we mostly focus on the rate problem 
(see Problem 2), which will be used in the application section (§ 6.2). 

2.1.1 Organization of the chapter 

This chapter is structured as follows. In Section 2.2, we introduce a formal 
definition of balanced sequence and we present an overview on their properties. 
Section 2.3 defines the constant gap sequences and addresses the problem of 
finding the rates admissibles for constant gap sequences. Section 2.4 shows 
how balanced sequences can be characterized using constant gap sequences. 
Section 2.5 addresses the problem of finding balanceable rates for several 
cases. 



2.2 Balanced sequences and bracket sequences 

Let A be a finite alphabet, A z is the set of bi-infinite sequences and A n is 
the set of infinite sequences defined on A. 

If u £ A z or u £ A n , then a word W of u is a finite subsequence of 
consecutive letters in u: W = u a u a +i ■ ■ -u a +k-i- The integer k is the length 
of W and will be denoted \W\. 

If a e A, \W\ a is the number of o's in the word W. 

Definition 4. The sequence u £ A Z U A n is a-balanced if for any two words 
W and W in u of same length, 
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-1 < \W\ a ~ \W'\ a < 1. 



The sequence u is balanced if it is a-balanced for each a £ A. 

If a e A, and u e ,4 N , we also define the indicator in a u of the letter a as 
the function l a (u) : A N — ► {0, 1} N by, l a (it)i = 1 if tt(i) = a and otherwise. 
The definition is similar in the bi-infinite case. The support in u of the letter 
a is the set S a = {i £ N : l a (u)i = 1}. 

As usual, for any real number x, [x\ will denote the largest integer smaller 
or equal to x and \x\ will denote that smallest integer larger or equal to x. 

Lemma 11. If a sequence u £ A n (resp. u € A z ) is balanced, then for any 
a € A, there exists a real number p a , such that 



p a is called the rate (or slope) ofl a {u). 

Proof. Let us define c n = SlLo l«(' tt )j an d remark that c n + c m — 1 < c n+m < 
c n + c m + 1. The rest of the proof is classical by sub-additivity arguments. 
The proof for {l a {u) n }n<o is similar. The fact that both limits coincide is 
obvious. □ 

Note that the sum of the rates for all letters in a sequence u is one. 



Now, we can present the main result which is the foundation of the theory of 
balanced sequences. We follow the presentation given in [109] and [85]. 

We now extend the definition of bracket sequences, as given in Definition 

2. 

Definition 5. A sequence u € {0, 1} Z (or u € {0,1}^ is bracket (resp. 
ultimately bracket) if there exist two real numbers 6 and p (and an integer 
k, respectively) such that for all n € Z (or n € (resp. n > k), u n = 
[{n + l)p + e\ - [np + 9\. 

Note that u n = [(n+l)p+6\ — [np+6\ is equivalent to S\ = { \n^+(f>] } n ez 
with <j)=l — 0/p, where \x~\ denotes the smallest integer no smaller than x. 

The following two theorems show the relation that exists between bal- 
anced sequences and bracket sequences. 

Theorem 11 (Morse and Hedlund). A sequence a € {0, 1} Z is balanced 
with asymptotic rate < p = 1/a for letter 1 if and only if the support S\ of 
a satisfies one of the following cases. 
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( ajirrational case ) p is irrational and there exists <j> £ ffi such that 

Si = { [ia + 4>\ } ie z or Si = {[ia + </>~|}i e z. 
(bjperiodic case) p £ Q and there exists <f> £ Q such that 

Si = {[ia + 4>\}iex. 
(cjskew case) p £ Q (p = k/n, k, n £ N) and there exists m £ Z such that 
Si = {[in/k + m\} i<k U {[in/k- l/fc + mj} i>0 

or 

Si = { [in/ k + raj } i>0 U { [in/ k - 1/k + raj } i<fc 

Theorem 12 (Morse and Hedlund). A sequence a £ {0, 1} N is balanced 
with asymptotic rate < p = 1/a for letter 1 if and only if the support Si of 
a satisfies one of the following cases. 

( ajirrational case ) p is irrational and there exists 6 el such that 

Si = { [ia + 6\ } im or Si = {[ia + 0"|} iGN . 
(bjperiodic case) p £ Q and there exists 8 £ Q such that 

Si = {[ia + 6\} ieN . 

As for the case p = 0, it includes two balanced sequences, namely the 
sequence where Si is the empty set and the sequence where Si is a singleton. 

The case for bi-infinite and infinite binary balanced are subtly different. 
Indeed, the skew case does not appear in the infinite case while cannot be 
overlooked in the bi-infinite case. 

The irrational case is the easiest case and can be characterized (see The- 
orem 13). The rational cases are more difficult to study. 

Note that the skew sequences are not periodic but are ultimately periodic. 
In our applications (see sections 6.2 and 6.3), we will mainly be using infinite 
sequences, so that the skew case will not be considered. However, even in the 
bi-infinite case, we have the following property. 

Proposition 1. Let u £ A z . 

(i) - If l a {u) is bracket for all a £ A, then u is balanced. 

(ii) - If u is balanced, then l a {u) is ultimately bracket for all a £ A. 

Proof, (i) is straightforward. 

(ii) is a direct consequence of Theorem 11, since in all three cases, the se- 
quence l a {u) is ultimately bracket. An elementary proof of (ii) which does 
not use Theorem 11 can be found in [106]. q 

Infinite balanced sequences with irrational rates are aperiodic and are 
sometimes called Sturmian words [94, 84]. 
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2.3 Constant gap sequences 

Constant gap sequences are strongly balanced sequences, in the following 
sense. 

Definition 6. A sequence G is constant gap if for any letter a, l a (G) is 
periodic, with a period of the form • • • 010 • • • 0. 

Note that this explains the fact that G is said to have constant gaps for 
the letter a, since each a is separated from the next a in G by a constant 
number of letters. 

Proposition 2. Constant gap sequences are balanced. 

Proof For each letter a, l a (G) is of the form (0"10 m )'". Therefore, l a (G) 
is bracket with p = l/(m + n + 1) and 6 = m/(m + n + 1). Using the 
characterization of balanced sequences given in Theorem 1, this shows that 
G is balanced. rj 

Proposition 3. Constant gap sequences are periodic. 

Proof. For each letter a, l a {G) is periodic with period p a . The period of G 
is lcm(p a , a £ A). □ 

In the next lemma, we give a characterization of constant gap sequences 
that stresses the fact that constant gap is some kind of strong balance. 

Proposition 4. G is constant gap if and only if, for any two finite words, 
W and W included in G with \\W\ — \W'\\ < I, then for each letter a, 
\\W\ a -\W'\ a \<\. 

Proof. Let a be a letter in the alphabet. 

First, assume that G is constant gap. If \W\ a — \W'\ a > 2, then, necessarily, 
\W\ — \W'\ > 2. Conversely, let W = alia and W = all' a be any two words 
in G with no a in the subwords U and U'. If \U\ > \U'\ + 1, then we have 
\\U\ - \W'\\ < 1 and \\U\ a - \W'\ a \ = 2. This is a contradiction. Therefore 
\U\ = \U'\ and G is constant gap. rj 

Since a constant gap sequence is balanced, each letter appears with a 
given rate in the sequence, note however that since a constant gap sequence 
is necessarily periodic, the rate of each letter is rational. 

As we will do in Section 2.5 for the case of balanced sequences, we now 
address the following question: 

Problem 1. Given a set (pi, • • • ,Pk), with pi + ■ ■ ■ +Pk = 1 is it possible 
to construct a constant gap sequence on K letters with rates (pi, • • • ,Pk)?- 
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We will not solve this problem for a general K, since it is NP-complete 
(see [26]). 

Remark 4- One can use the following facts to show that the set of rates 
solving 1 is finite. Let (pi, ■ ■ ■ ,px) be a set of rates solving problem 1. We 
order the rates so that 1 > p\ >■■■> Pk- Then necessarily, 

— For all i, p^ 1 is an integer. 

- For all i, Pi _! >p t > 

Note that combining both items leaves only a finite number of possibilities 
for (pi ■ ■ ■ ,p K ). 

We will now give some properties of the set (pi, ■ ■ ■ ,Pk) which will be 
useful in the following. A characterization of such (pi,--- ,Pk) is given in 
[115], under the name exact covering sequences, but it does not provide an 
algorithm. 

Definition 7. The set of couples {(0i,gi),i = 1-K} is called an exact 
covering sequence if for every nonnegative integer n, there exists one and 
only one 1 < i < K such that n = 6i mod 

As a general remark, note that (pi,--- ,Pk) are rational numbers of 
the form pi = ki/d, with d the smallest period of G. Therefore, we have, 

ki/d = 1 and for each i, ki divides d. By definition of the rates, we also 
have p a = l/q a for all letters. 

We have the following result. 

Proposition 5. The rates (pi,-- - ,Pk) are constant gap if there exists K 
numbers called phases, 0\,---9k such that the couples {(0i,qi),i = 1- K} 
form an exact covering sequence. 

Proof. This property is a simple rewriting of the fact that each letter a; in a 
constant gap sequence appears every {6i + kqi, k € N}. rj 

Now, suppose that i = 1- K} is an exact covering sequence. 

Then in the series 

^) d = ££** +fcw , 

i=l fc>0 

the coefficient of x n in this series is equal to 1, Vn > 0. Therefore, we have 

K e i 
s t x \ = x i = 1 

w ^ 1 - xi* 1-x' 

i=l 

Using this characterization we have the following interesting property 
which was proved in [93]. 
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Lemma 12. Assume {(6i, qi), i = 1 ■ ■ ■ K} is an exact covering sequence and 
that P = maxi Then P appears at least twice in the set qi, ■ ■ ■ ,qx- 

Proof. The proof given here is similar to the discussion in [115] on exact 

covering sequences. Let w ^= e 2z7r / r f or some integer r > 1. By definition, w 
is a primitive r-th root of one. We have: 

K i \ e 

I \al \ S^( w - x ) x i W-X 

(w-x)s(x) = y — — = - — . 

v ' w ^ 1 - x* 1-x 

i=l 

Let x — ► w. This yields 

E^^T=0- (2-1) 

z:r\qi 

Now, take r = P. The set {i : P\qi} is exactly the set {i : P = qi}. 
Equation (2.1) specified for r = P can be written 

i:qi=P 

This implies that the set {i : P = qi} cannot be reduced to a single point 
since w is not zero. rj 

To give some concrete examples, we consider the cases where K is small. 
First, note that in the case where the ki are not all equal, (assume fci is the 
largest of all), we have 

y^kj/h = d/ki - 1 = h, 

where l\ is the gap between two letters a\. This implies, 

h < K - 2. (2.2) 

Proposition 6. There exists a constant gap sequence G with rates (p, l—p) 
if and only if p = 1/2. 

Proof. Let a be a letter in G with gap /. Since the alphabet contains only 
two letters, / = 1. This means p = 1/2. rj 

Proposition 7. There exists a constant gap sequence G with rates {pi,P2,P3) 
if and only if (pi,P2,P3) € {(1/3, 1/3, 1/3), (1/2, 1/4, 1/4)} (up to a permu- 
tation). 

Proof. Assume that {pi,P2,P3) ^ (1/3,1/3,1/3) (otherwise, G = (abc) u is 
constant gap). Using Inequality (2.2), l\ = 1 and p\ = 1/2. Therefore, the 
sequence obtained from G when removing all the letters oi is constant gap. 
Applying Lemma 6 shows that P2 = Pz- The only solution is pi = 1/2, p 2 = 
1/4 and p% = 1/4. The associated constant gap sequence is (01020103)". q 
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Proposition 8. A constant gap sequence G with rates (pi,P2,P3,P4) exists 
if and only if (pi,P2,P3,P4) belongs to the set (up to a permutation), 

{(- ---)(----)(----)(--- -)} 

^4' 4' 4' 4 M 2' 4' 8' 8 M 2' 6' 6' 6 M 3' 3' 6 V' 

Proof. We give a sketch of an elementary proof of this fact. If the rates are 
all equal, then (pi , P2 , P3 , Pa) = (1/4, 1/4, 1/4, 1/4). Now, note that Equation 
2.2 implies that if the rates are not all equal l\ < 2. We consider any letter 
a>i,i ^ 1, assume that the number of oi's in between two o^'s is not con- 
stant and takes values m and n, m > n. then we have h > (n — l)h + n 
on one hand and k < n(h) — 3 on the other hand. This is impossible 
since h < 2. Therefore, the number of ai's in between two o^'s is con- 
stant. This is true for all i. The sequence formed by removing all Oi's is 
still constant gap. It has rates of the form (1/3,1/3,1/3) or (1/2,1/4,1/4). 
From this point a case analysis shows that the original sequence has rates 
(1/2, 1/4, 1/8, 1/8), (1/2, 1/6, 1/6, l/6)or(l/3, 1/3, 1/6, 1/6)} by inserting the 
letter oi in a constant gap sequence over the letters 02,03,04. □ 

These few examples of constant gap sequences illustrate the fact that 
there are very few rates that can be achieved by constant gap sequence. 

2.4 Characterization of balanced sequences 

Several studies have been recently done on balanced (or bracket) sequences 
[83, 107, 106, 109, 90, 89]. In [55, 75], a characterization involving constant 
gap sequences is given. 

The proof of the following results, Proposition 9 and Theorem 13 was 
given by Graham [55] for bracket sequences. An independent later proof can 
be found in [75] for balanced sequences. The relation between balanced and 
bracket sequences given in Theorem 1 makes both proofs more or less equiv- 
alent. 

Proposition 9. Let U be a balanced sequence on the alphabet {0,1}. Con- 
struct a new sequence S by replacing in U, the subsequence of zeros by a 
constant gap sequence G on an alphabet A\, and the subsequence of ones by 
a constant gap sequence H on a disjoint alphabet A2 ■ Then S is balanced on 
the alphabet A\ U A2 ■ 

Proof. We give a proof similar to Hubert's proof ([75]). Let o be a letter in 
A\ (the proof is similar for a letter in „4 2 )- Let W and W be two words of 
S of the same length. Then, the corresponding words X and X' in U verify 
\\W\o — |W^'|o| < 1 since U is balanced. If we keep only the 0's in X and X', 
then the corresponding Z and Z' words in G satisfy \\Z\ — \Z'\\ < 1. Since G is 
constant gap, and using Lemma 4, ||Z| a — |Z'| a | < 1. We end the proof noting 
that the construction of Z and Z' implies \Z\ a = \W\ a and \Z'\ a = \W'\ a . q 
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Conversely, we have the following theorem. 

Theorem 13. Let u £ A z be balanced and non ultimately periodic. Then 
there exists a partition of A into two sets A\ and Ai such that the sequence 
v defined by: 



is balanced. Furthermore, the sequences z\ and z-i constructed from u by keep- 
ing only the letters from A\ and A2 respectively have constant gaps. 

2.5 Balanceable rates 

Let us formulate precisely the problem which we will study in this section. 

Problem 2. Given a set (pi, • • • ,Pk), is it possible to construct a balanced 
sequence on K letters with rates (pi, • • • ,Pic)? 

We will see in the following that this construction is not possible for all 
values of the rates (pi, • • • ,Pk)- If a if-tuple (pi, ■ ■ ■ ,Pk) makes the con- 
struction possible, such a tuple is said to be balanceable. A similar problem 
has been addressed in [55, 90, 107, 46], where relations between the rates of 
balanced sequences are studied. 

2.5.1 The case K = 2 

This case is well known and balanced sequences with two letters have been 
extensively studied (see for example [32, 85]). The following result is known 
under many different forms. 

Theorem 14. For allp, < p < 1, the set of rates (p, 1 — p) is balanceable. 

Proof. The proof is similar to the proof of the first part of Theorem 1. We 
construct a sequence S as the support of the function s(n) = [pn\ — [p(n— 1)J . 
S is a balanced sequence because the interval ]k, k + m] contains exactly 
e = [pk + pmj — [pfcj elements of S. Now, [pmj + [pfcj — [pfcj < e < 
\pm~\ + \j)k\ — [pk\ . This shows the value of e can differ by at most one when 
k varies so S is a balanced sequence. If S' is the complementary set of S, 
then it should be clear that S' has asymptotic rate 1 — p and S' is balanced 
because S^ k k+m ^ contains m — e elements. rj 

Note that the proof of Theorem 14 also gives a construction of a balanced 
sequence with the given rates. 



v n = l ifu n e A\ 
v n = ifu n € A 2 



(2.3) 
(2.4) 
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2.5.2 The case K = 3 

The case K = 3 is essentially different from the case K = 2. In the case 
K = 2, all possible rates are balanceable while when K = 3, there is only one 
set of distinct rates which is balanceable. This result, when formulated under 
this form, was partly proved and conjectured in [83] and proved in [107]. 
In earlier papers by Morikawa, [90], a similar result is proved for bracket 
sequences. If Theorem 1 is used, then the result of Morikawa can be used 
directly to prove the following theorem. 

Theorem 15. A set of rates {pi,P2,P3) is balanceable if and only if, 
(pi,P2,Ps) = (4/7,2/7,1/7) 

or two rates are equal. 

Proof. The proof of Morikawa is very technical since it does not use the 
balanced property for bracket sequences. If the balanced property is used, 
then the proof becomes easier. We give a proof slightly simpler than the 
proof in [107]. First, assume that p\ = P2- Then, let S be a balanced sequence 
with two letters {a, 6} constructed with the rates (pi +Pi,P3). In S, replace 
alternatively the "a"s by the letters oi, d2, we get a sequence S' on alphabet 
{01,02,6} with rates {pi,Pi,P3). Let us show that S' is balanced. Since S is 
balanced, the number of "o"s in an interval of length m is k or k + 1, for 
some k. Now, for S', the number of "oi"s (resp. 'Vs) in such an interval is 
either (k - 1) /2 or (k + 1) /2 if k is odd and k/2 or k/2 + 1 if k is even. This 
proves that S' is balanced. 

Now, assume that (pi,P2,P3) are three different numbers. We assume that 
Pi > Pi > P3- We will try to construct a sequence W with these respective 
rates on the alphabet {a,b,c}. 

step 1: the sequence "aca" must appear in W. 
There exists a pair of consecutive "o" with no "b" in between since pi > P2 ■ 
This means that a sequence "00" or "aca" appears. If "ao" appears, then a 
"c" is necessarily surrounded by two "a"s. 

step 2: the sequences "baab" and "abaaba" must appear in W. 
There exist a pair of consecutive "b" with no "c" in between. This sequence 
is of the form "ba n b" . Now, n < 1 is not possible because of the presence 
of "aca" and 6-balance. n > 3 implies the existence of u a n ^ 1 ca n ^ lv by a- 
balance which is incompatible with u ba n b v because of 6-balance. Therefore 
n = 2. Note that this also implies the existence of "aa" and of u abaaba v . 

step 3: the sequence "abacaba" appears in W. 
the sequence W must contain a "c". This "c" is necessarily surrounded by 
two "a"s since "aa" exists by step 2. This group is necessarily surrounded 
by two "6"s since "baab" exists, and consequently, necessarily surrounded by 
two "a"s because "abaaba" exists. We get the sequence "abacaba" . 
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Last step: W = (abacaba)" . 
No letter around this word can be a "c" because "6aa6" exists. None can 
be a "6" since "oca" exists. Therefore, they have to be two "o"s. Then note 
that the two surrounding letters cannot be "c" (because of the existence of 
"abaaba"), nor "a" (because of the existence of "6ac") so they are "6", then 
followed necessarily by "a" (because "aa" exists). At this point, we have the 
sequence "*abaabacabaaba-k v . Both *s are necessarily "c"s. 

To end the proof, note that we have obtained the configuration around 
every "c" and this determines the whole sequence. The sequence W is periodic 
of the form (abacaba) w . rj 

2.5.3 The case K = 4 

For distinct rates, the case K = 4 is very similar to the case K = 3. However 
when two rates are equal, this case is more complicated. Again, a similar result 
for bracket sequences is contained in [90] under a weaker form since, the proof 
is only done for rates of the form (oi/c, ••• ,04/c). The following theorem 
gives the result in its full generality. Again, using the balanced property 
helps to keep the proof rather elementary. 

Theorem 16. A rate tuple {pi,P2,P3,P4) with four distinct rates is balance- 
able if and only if (pi,P2,P3,P4) = (8/15, 4/15, 2/15, 1/15). 

Proof. We suppose that pi > P2 > P3 > Pi and we show that there is only one 
balanced sequence with frequencies pi > P2 > Pz > Pi and those frequencies 
are (8/15,4/15,2/15,1/15). 

As a preliminary remark, note that if Pi > pj , then there exists at least 
one word "a; •••a;" that does not contain any aj. This fact will be used 
several times in the following arguments. 

The proof involves different steps. 

Step 1: W contains the words "aca" or "ada" or "acda" or "adca". 

There exist two consecutive "a" s with no "6" in between because p\ > P2 ■ 
Therefore, either "00" or "oco" or "ado" or "acda" or "adca" exist. If "aa" 
exists, then, a "c" is surrounded by two "a"s. 

Step 2: W contains the word "baab" 

First, we show that if a word "6a™6" exists, then n = 2. Indeed, step 1 
makes "66" and "6a6" impossible. On the other hand, if n > 3, the existence of 
u a n ^ 1 ca n ^ lv is necessary by a-balance and is incompatible with the existence 
of "6a n 6" because of 6-balance. 

Now, if no word of the form "6a n 6" exists, then there exist two consecu- 
tive "6"s with one "d" and no "c" in between. Such a word is of the form: 
"baida k b v and is surrounded by some "a"s, so that we get a word called s\ 
and equal to "a l ba : >da k ba lv . Note that the numbers i, j, k, I may be equal to 
zero but j + k > 1 by 6-balance. 

There also exist two consecutive "c"s with no "d" in between. In between 
those two "c" we must have some "a"s and some "6"s. In fact we have exactly 
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one "6" since no word ll ba n b" exists by assumption. We pick such a word 
called S2 which is of the form: "ca m ba n c v . Note that i, j, k, I, m, n are integers 
that can differ by at most one. As for the length of si, we have > 
max(4, 2{n + m) + 1) > n + m + 3 = \sz\. This is impossible by c-balance. 

step 3: the word "abacabaabacaba" exists in W. There exist two consecu- 
tive "c"s with no "d" in between. From step 3 in the proof of Lemma 15, we 
know that a "c" is necessarily surrounded by the word "aba" . Moreover, from 
step 4 in the proof of Lemma 15, we have: " 'abacabaabaU caba" , where U is a 
word that contains no "cf and no "c" . U cannot start with an "a" (because 
of "bacab") and cannot start with a "6" (because of "oco"). Therefore U has 
to be empty. 

Step 4 : W is uniquely defined and is periodic of period "abacabadabacaba". 
Somewhere, W contains a "cf. From this point on, we can extend the word 
uniquely around this "cf as the word: "abacabadabacaba" , and, on the other 
hand, the word "abacabaabacaba" has to be surrounded by two "cf's. This 
ends the proof. rj 

To complete the picture, it is not difficult to see that, 

Proposition 10. if the tuple {pi , P2 , P3 , P4) is made of no more than two 
distinct numbers, then it is balanceable. 

Proof. First, if the rates are all equal, they are obviously balanceable. If 
three of them are equal, say pi = P2 = P3, then, we can construct a balanced 
sequence with rates {3pi,P4) and we construct a balanced sequence with 
rates (pi,Pi,Pi,P4) by using Proposition 9 (where we take G the constant 

gap sequence (aia2«3) w and H ^= (04)"). If two pairs of rates are equal, 
say p\ = P2 and p% = P4, then we construct a balanced sequence with rates 
(2pi,2p3) and we apply Proposition 9. 

□ 

If the tuple (pi,P2,P3,P4) is made of exactly three distinct numbers, then 
this is a more complex case which is not studied here. 



2.5.4 The general case 

In this section, we are interested in the case of arbitrary K. First, note that 
Proposition 10 easily generalizes to any dimension. 

Proposition 11. If the tuple (pi,P2,P3, ■ ■ ■ ,Pk) is made of less than two 
distinct numbers, then it is balanceable. 

Proof. The proof is similar to the proof of Proposition 10. □ 

Proposition 12. If the tuple (pi,P2,-" >Pk) is balanceable, then the tuple 
(pi /k, ■ ■ ■ , pi /k, P2 , ■ ■ ■ ,Pk) is balanceable. 



50 2 Balanced Sequences 



Proof. The proof is very similar to that of Proposition 9. If W is a bal- 
anced sequence with letters {oi,-- - ,cik}, consider the sequence W con- 
structed starting from W and replacing each "oi" by an element of 
"62", • • • , "6fc) in a cyclic way. Note that W has the following set of rates, 
{Pi/k, ••• ,Pi/k,p 2 ,--- ,Pk)- 

Next, we show that W is balanced. Since W is balanced, for an arbitrary 
integer m, the number of "ai"s in an interval of length m is n or n + 1, 
for some n. Now, for W, the number of "6/'s in such an interval is either 
|_(n — l)/fcj or + l)/fcj. This proves that W is balanced. rj 

For the general case and distinct rates, it is natural to give the following 
conjecture (due to Fraenkel for bracket sequences): 

Conjecture 1. A set of distinct rates {pi, ■ ■ ■ ,Pk} with K > 2 is balanceable 
if and only if 

{Pi, ■■,Pk} = {2 k - 1 /(2 k - 1), • • • , 2 K - i /(2 K - 1), • • • , 1/(2 K - 1)}. 

We have not been able to prove this fact. Morikawa has also given some 
insight in this conjecture. Very recently, in [108] the cases K = 5 and K = 6 
are proven using techniques inspired by those introduced here. However, it 
seems clear that a different approach is needed in order to complete the proof 
in the general case. Here, we only have partial results given in the following 
propositions. 

Proposition 13. The rates (2 K - 1 j{2 K -1), • • • , 2 K - i j{2 K -\), • • • , 1/(2 K - 
1)) are balanceable, for all K e N. 

This proposition is the "if" direction of the conjecture. 

Proof. We construct a balanced sequence wk in the following inductive way. 
w\ = 01, wk = WK-idK^K-i and wk = {vk) u ■ First note that wk has 
rates {2 K - 1 j{2 K - 1), • • • , 2 K - i /(2 K - 1), • • • , 1/{2 K - 1)). Then, we show 
that wk is balanced by induction. In the sequence wk, any letter (say letter 
j) appears 2 K ~ J times in one period and is of the form of 2 K ~i — 1 intervals 
of the same length (2 J ) and one of length 1? — 1 . 

By construction of Wk+i , these properties still hold and therefore, Wk+i 
is balanced. rj 

Proposition 14. Let K > 2 and w be balanced with rates p\ > ■ ■ ■ > pk, 
then, w is ultimately periodic. In particular, this means that pi e Q, VI < 
i<K. 

Proof. If w is not ultimately periodic, Theorem 13 says that w is composed 
of two constant gap sequences. At least one of these sequences has at least 
two letters, and therefore two letters have rates which are equal by Lemma 
12. Therefore, the rates in w of these two letters are also equal. n 
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Proposition 15. Let w be balanced with rates p\ > ■ ■ ■ > px, with the 
following property: for any 1 < i < K , there exists two consecutive letters 
"cii" with no cij in between, with j > i. Then, (pi, ■ ■ ■ ,Pk) = (2 K ~ l /(2 K — 
1), • • • , 2 K ~ l /(2 K — 1), • • • , 1/(2 K — 1)) and w is uniquely defined, up to a 
shift. 

This proposition is a partial "only if" result for the conjecture. 

Proof. The proof holds by induction. Let Vk denote the period of the balanced 
sequence with rates (2 k - 1 /(2 k - 1), • • • , 2 k - i /(2 k - 1), • • • ,l/(2 fc -l)) given 
in Proposition 13. We recall that according to the construction in the proof 
of proposition 13, Vk = Vk-idt-Vk-i- We will prove by induction that w is 
periodic with period Vk- 

We prove by induction on k that w contains "vk-iakVk-iVk-idkV 1 ^.^ and 
that for j > k, each letter in w, "a/', is surrounded by Vk-i's, for all possible 
1 < k < K . For the first step of the induction (k = 1), note that according to 
the property on w, w contains the factor "aioi" which is the same as "vivi" . 
Therefore any other letter is surrounded by two "oi"s. This also implies the 
existence of "01O2O1O1O2O1" by using a similar argument as step 2 in the 
proof of Theorem 16. This ends the case k = 1. 

For the general case, by the induction assumption, at is surrounded by 
Vk-2, and w contains the word u Vk-2(ikVk-2 v ■ The existence of the factor 
"^fc-20fc-i^fc-2^fc-20fc-i^fc-2" proves that this word is surrounded by two 
"ofc_i"s. Therefore, two consecutive Ofc form the word 

Vk-2ak-iVk-2akVk-2ak-iVk-2Uvk-2ak-iVk-2akVk-2(ik-iVk-2, 
where U does not contain any letter "oj" , j > k. 

— If U does not contain "afc_i" and contains a letter "0^' with i < k — 1 then 
U is reduced to this letter, because of Ofc_i -balance and the existence of the 
word Ofc_i^fc_2i'fc-20fc-i (induction assumption). But now the construction 
of Vk-2 implies the presence of "o^i_iOi" and contradicts the existence of 

"diVi-iVi-id" . 

— If U contains "ofc_i", then by Ofc_i-balance, U must contain Vk-2dk-iVk-2. 
therefore U is of the form Xvk-2(ik-iVk-2Y . The arguments used for 
U can be applied to X and Y . If they are not empty, they must both 
contain Vk-2<ik-iVk-2- Eventually, we have the existence of the word 
Vk-2Vk-2<ik-iVk-2Vk-2 which contradicts obvioulsy the existence of the 
word t'fc-20fc-i^fc-20fci'fc-20fc-i^fc-2 by 01-balance. 

Therefore, U is empty. This implies the existence of Vk-idkVk-iVk-icikVk-i 
Now, we finish the proof by noticing that the letter dx is surrounded by 
Vk-i and by noting that Vk-iVk-i is necessarily surrounded by "ajf". q 

Proposition 16. The projection w' of a sequence w over the alphabet A— {0} 
is w where all d's have been removed. Ifp a > 0.5, then w is balanced implies 
that w' is balanced. 
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Proof. Choose two words v[,v' 2 of length n in w' . Let v\ and v 2 be any 
two words in w whose projections over the alphabet A — {a} are v[ and v' 2 , 
respectively. Assume, furthermore, that the first and last letters in v\ and v 2 
are not a. Let k = \vi \ — n and / = \v 2 \ — n denote the number of appearances 
of the letter a in V\ and v 2 , resp. 

Step 1: If I = k then the difference in the number of occurrences of any 
letter b ^ a in v[ and in v' 2 is at most 1, since w is balanced, and since the 
number of 6's in v\ (resp. V2) is the same as its number in v[ (resp. v' 2 ). 

Step 2: Assume that / > k + 1. 

- Let v 2 be the word obtained from v 2 by truncating the first and last letter. 
Then \v 2 \ = n + I — 2, and the number of o's in v 2 is /. 

- Let v\ be the word obtained from v\ by concatenating to it the next m = 
l—k — 2 letters that appear after v\ in the sequence w. Then \vi \ = n + l — 2 = 
\v 2 \, and the number of a's in ii is not larger than k + m = I — 2. This is a 
contradiction with the fact that w is balanced. 

Step 3: It remains to check the case / = k + 1. Add to v\ the next letter 
that occurs in w to its right, to form the new word v\. If it is not a then we 
have two successive letters that are not a, which contradicts the fact that a 
has an asymptotic frequency of at least 1/2. If it is a, then v\ and v 2 have 
the same number of o's. We can now apply the same argument as in step 1 
and conclude that the number of occurrences of any letter b in v[ and in V 2 
differs by at most 1. 

Combining the above steps, we conclude that w' is balanced. rj 

2.5.5 Extensions of the original problem 

So far we have only analyzed the case where all the rates add up to one. The 
different results tend to prove that very few sets of rates are balanced. 

Now let us look at a generalization when all the rates do not add up to 
one. Assume that S is a sequence on the alphabet {01, a 2 , ■ ■ ■ , ak, *}• We only 
require that S is balanced for the letters a\ , ■ ■ ■ ,a>k, but not for the special 
letter *. 

On a more practical point of view, the question can be viewed as whether 
this allows more possibilities for rates to be balanced when "losses" are al- 
lowed (represented by the letter *). Then again, in general, the rates are not 
balanced, even if the total sum is very small as illustrated by the following 
proposition. 

Proposition 17. For an arbitrary e > 0, there exists two real numbers pi 
and p 2 such that p\ + p 2 < e and such that there is no sequence S on the 
alphabet {a, b, *} with asymptotic rate pi for letter a and p 2 for letter b which 
is balanced for a and b. 

Proof. Choose two irrational numbers p\ and p 2 with pi + p 2 < e such that 
Pi , p 2 and 1 are not linearly dependent on Z . Now assume that there exists a 
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sequence S on {a, b, *} with asymptotic rate p\ for letter o and P2 for letter 
b which is balanced for o and b. By Theorem 1, then there exists two real 
numbers x,y such that l a (S)(n) = 1 if x + p\n mod 1 € [1 — pi, 1] and 
otherwise. lf,(5)(n) = 1 if y + p^n mod 1 € [1 — P2, 1] and otherwise. In 
the cube [0, l] 2 , the set of points (x, y) + n(pi,p2) mod (1, 1) is dense (see 
for example, Weyl's ergodic theorem [102]) and therefore hits the rectangle 
[(1 — pi, 1 — P2), (1, 1)]. This is not possible. q 

More on this kind of problems can be found in [106]. 

To end this short overview on balanced sequences, we must mention on 
the positive side that "usual" rates, such as (1/fc, 1/fc, - - - ,1/fc) are often 
balanceable. In Appendix 2.6, some examples of balanced sequences and their 
rates are given. 

2.6 Appendix 

In this appendix, we shall give a collection of balanceable set of rates which 
can be put into two classes. Some of them are composite: they can be con- 
structed using Proposition 12 (once or several times) starting from a smaller 
balanceable set. The ones which cannot be constructed that way are called 

primitive. 

We shall first give a list that contains some primitive balanceable rates 
with K = 4 (as well as other cases with different values of K). 

— (1/11,2/11,4/11,4/11) is balanceable and S = (abcababcabd)" . 

— (1/11,2/11,2/11,6/11) is balanceable and S = (abacaabacad)" . 

— (1/11,1/11,3/11,6/11) is balanceable and S = (acabaabadab) w . 

— for all K, ((2 K - l /{2 K - 1),-- - ,2 K - i /{2 K - 1),--- ,1/(2 K - 1))) is bal- 
anceable. The associated balanced sequence is constructed recursively as 
in Proposition 13. 

Here are other balanceable sets of rates when K = 4 which are composite. 

— (1/14,1/14,4/14,8/14) is balanceable and S = (abacabaabadaba)" . It is 
composite since it comes from (1/7,2/7,4/7) where the smallest rate is 
split into two. 

— For each real number < p < 1, the rates (1 —p,p/4,p/4,p/2) are balance- 
able, with a corresponding balanced sequence constructed from a bracket 
sequence with rate p where all 1 are replaced in turn by the sequence 
(abac)" and each by the letter d. It is composite, originating from (1— p, p) 
and split twice. 

— (1/fc, • • • , 1/fc) is balanceable. A balanced sequence is: S = (010203 • • • auY ■ 
This is composite, coming from (1) split once into fc rates. 
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— (p, ■ ■ ■ ,p,p,--- , j3) is balanceable. A balanced sequence with those rates is 
constructed in the following way: Choose a balanced sequence S on letters, 
(A, B) with rate (pi = Yli=i V1V1 = Yli=i P)- ^ n & replace all the A (resp. 
B) by ai,a2,-"" i a fc (resp. , 6^) in a round robin fashion to get a 

balanced sequence with the required rates. This is also composite, coming 
from (pi , P2 ) , where p\ is split into k rates and P2 is split into h rates. 
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3.1 Introduction 

This chapter introduces the timed Petri net models with a special focus on 
timed event graphs with stochastic timings. 

The first goal on the chapter is to give a precise definition of the semantics 
of timed Petri nets with timings associated to the transitions. This is used to 
show that the evolution of the firing epochs in an event graph can be written 
under the form of a max, plus linear system, as proved in [23]. 

We give several examples from queuing theory, manufacturing systems 
and communication networks of event graph models in order to illlustrate 
their modelling power as well as their limitations. 

The (max, plus) evolution equations are used to derive a vectorial form 
of Lindley's equation for waiting times and sojourn times in networks of 
queues [25]. This equation will be used in Chapter 4 to show that the sojourn 
time of a customer in a timed event graph is a multimodular function of the 
admission actions. Thus, timed event graphs will constitute the most general 
class of networks treated in this book for which the general theorems proved 
in Chapter 1 will apply. 

3.1.1 Organization of the chapter 

This chapter is organized as follows. Section 3.2 introduces the definitions 
and notations foir stochastic Petri nets. Section 3.3 deals with the dynamic 
behavior of event graphs and shows that the evolution equations of event 
graphs can be put under the form of a linear equation in the (max,+) for- 
malism. Section 3.4 shows several examples of discrete event systems taken 
from queuing theory, manufacturing and telecommunication with their even 
graph modeling and the associated (max,+) evolution equation. Finally, Sec- 
tion 3.5 derives a vectorial Lindley's equation for (max,+) linear systems. 

3.2 Stochastic Petri Nets 

Petri nets constitute a model of discrete systems that combines concurrency 
and competition at the node level. They were primarily used for analyzing 
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logical properties of systems involving parallelism and synchronizations, see 
for example [44]. More recently, the notion of time has been introduced in 
Petri nets to make performance analysis possible as in [31, 23]. 

Definition 8. A Petri net is a bi-partite graph given by the tuple Q = 
(V, Q, £, Mo), where V = {pi, ■ ■ ■ ,pp} is the set of places, 
Q = {<7i , ■ ■ ■ , qQ } is the set of transitions, 

£ is a subset of (Q xV)U (V x Q) and is the set of edges. We will denote 
by p' ('p, q' , *q), the set of downstream (upstream) transitions (places) of 
place (transition) p (q). 
M° : V — ► {0, 1, 2, • • • M} is the initial number of tokens in each place. 

Since Q is an oriented graph, then the following definitions are classical. 

— Paths: a path is a set of transitions {qi, -- ,q n } and a set of places 
{fii • • • iPn-i} such that for all i = 1 ■ ■ -n — 1, (qi,Pi) and (pi,qi + i) are 
arcs in A. 

— Circuits: a circuit is a path such that q\ — q n . 

— Strongly connected component: a strongly connected component is a max- 
imal sub-net C of Q such that for each pair of transitions q\,qi in C, there 
exists a path in C from q\ to (fe. 

— inputs and outputs: an input transition q verifies *q is empty. An output 
transition q is such that q* is empty. Note that an input transition (resp. 
an output transition ) is a strongly connected component by itself. 



A Petri net is displayed in Figure 3.1. Places are drawn as circles and 
transitions as thin rectangles. The initial marking is displayed with tokens in 
the places. 

Definition 9. A timed Petri net is a Petri net with in addition, firing times 
associated with the transitions 1 

1 Time is sometimes also attached to places. It is possible convert a Petri net 
with time associated with places into a Petri net with time on transitions and 
vice-versa. 




Fig. 3.1. A petri net 
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— a = {a q (n),q £ Q,n £ N} is the set of the firing times of the transitions, 
where <r q {n) is the nth firing of transition q. 

In the following, we will assume that the stochastic processes {<7i(n)}„ e N 
are stationary with finite expectations. In addition, we assume that the se- 
quences {o~i(n)} and {<rj(n)} are mutually independent for all i and j in 
Q. 

The marking in place p is a right-continuous function M p (t) : R + — > N 
and evolves according to the semantics of a timed Petri net which follows. 

Enabling - A transition q is enabled at time t if each input place p of q (p £ *q) 
contains a token at time t. 

Firing - If a transition q starts its n-th firing at time t, then one token is 
"frozen" in each input places at time t. The transition ends the firing at time 
t + a q (n) while the frozen tokens are removed and one token is added in all 
output places of q at time t + a q (n). 

def 

The marking M(t) = (Mi(t), • • • Mp(t)) of a Petri net changes according 
to this firing rule: 2 if transition q fires for the n-th time at time t, 

(M p (t + a q {n)-)) - 1 if p e *q, 
M p (t + a q (n)-) + l if peg', (3.1) 
M p (t + a q (n)-) otherwise. 

Initial conditions - The initial marking is the marking at time 0, and no 
tokens are frozen yet. 

V P £V, M p (0_) = M p °. (3.2) 

Input transitions - Note that the firings of input transitions cannot obey 
the firing rule defined above. Input transition would fire at any time since 
they are always enabled. To overcome this difficulty, firing times of the input 
transitions are given as extra data. 

The sequence of firing epochs of input transition k is given beforehand 
and denoted {Uk{n)} ne jq. The sequence {Uk{n)} ne jq is non-negative and in- 
creasing, < Uk{l) < UkiX) < • • • < Uk{n) <■■■■, and Uk{n) is the epoch of 
the n-th firing of input transition k. At the same time, one token is released 
in each output place of k. 

These input transitions are very useful for modeling purposes. They model 
exogenous arrivals of tokens into an open system (see section 3.4 for some 
examples). 

Several pictures of the firing of a transition are displayed in Figure 3.2. 
At time 0_, the initial marking is displayed in Figure 3.2(a). At time 0, 



2 the notation f(x-), for a right continuous functions / is the limit of f(y) when 
y goes to x, y < x 






Fig. 3.2. Three snapshots of a Petri net taken at time (a) 0_ , (b) and o"i(l). 
Frozen tokens are grey, available tokens are black. 



transition 1 becomes enabled and freezes one token in each input place as 
it starts its first firing. The situation remains unchanged up to time o"i(l)_ 
(Figure 3.2(b)). At time oi(l), the frozen tokens are removed. Tokens are put 
in the output places of transition 1. At the same time, transitions 3 becomes 
enabled and starts firing by freezing the new token (Figure 3.2(c)). 
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3.2.1 Properties 

The behavior of a (timed) Petri net may be classified according to the fact 
that all transitions fire infinitely often, or some starve after some time or even 
everything stops in a situation where no transition can fire. 

More precisely, a Petri net is live if any transition q may eventually be 
fired starting from any marking M reachable from M° : there exists a sequence 
of transitions qi, ■ ■ -q n such that firing this sequence, starting in M yields a 
marking M' enabling q. 

A petri net is in a deadlock M if after reaching the marking M from M°, 
no transition is enabled anymore. 

A petri net with input transitions is in an input deadlock M if after reach- 
ing the marking M, no transitions is enabled except the input transitions 
(which are always enabled by definition). 

In [22], it has been shown that liveness (or presence of deadlocks) does not 
depend on the timings of the Petri net. This is a structural property which 
only depends on the topology, the routing of tokens in places with several 
output (see [48]) and the initial marking of the net. 

Checking if a Petri net is live is difficult in general. It has been proven 
co-NP-Complete for the class of Free choice nets (see [44]). 

However, testing if a Petri net is live is polynomial for some other sub- 
classes, such as state machines or event graphs (see § 3.2.2). 

3.2.2 Event graphs 

Definition 10. An event graph is a Petri net where each place has one in- 
coming transition and one out-coming transition: 

Vp&V,\'p\ = \p'\ = l. (3.3) 

The net displayed in Figure 3.1 is an event graph: all places have one 
incoming and one outgoing transition. 

The modeling power of an event graph is limited but its dynamic is simple 
and can be put in a linear from in the (max, plus) semi- ring (see § 3.3.3). 

This subclass of Petri net can be used to model several systems on com- 
munication, manufacturing or queuing which are highly synchronized and 
where the routes are all pre-defined. Some examples will be given in Section 
3.4. 

Liveness - As mentioned above, testing if an event graph is live is easy. 

Lemma 13. An event graph is live if and only if all circuits are marked 
under the initial marking. 

For a proof of this lemma, see for example [44] . This provides a polynomial 
test for liveness of event graphs. 
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Input connectedness - We consider an event graph with input transitions. We 
say that the graph is input-connected (i.e. for each transition q in the net, 
there exists a path from one input transition to q). 

For each transition (or node) q in Q, we consider all paths tt from input 

clef 

transitions to q. This set is denoted by V(q). We also denote by M°(tt) = 
J2 s eir -M°(s), the sum of the initial tokens on the path tt. Now, we define 

L(q) = min M{tt). (3.4) 

TtEV(q) 

Lemma 14. For an event graph which is input connected, the n + L(q)-th 
firing of transition q of Q involves a token produced by the n-th firing of an 
input transition to which q is connected. 

Proof. Let h q be a shortest path from one input transition qo to q with L(q) 
tokens. The length of h q is called the "distance" from qo to q. The proof holds 
by induction on the length of h q . If h q = 0, then q = q and the result is true. 
Suppose that the result is true for all transitions at "distance" k — 1 from q • 
Choose q at distance k, then the transition q' preceding q on the path h q is 
at distance k — 1 from q and induction applies to q' . Now the place (q',q) 
contains m tokens. By definition of q', L(q') = L(q) — m, and by induction, 
the n + L(q')-th firing of transition q' uses the token number n and since the 
buffer place between q' and q is only fed by q' and emptied by q by the event 
graph assumption, the n + L(q)-th firing of q will use the same token (n-th 
token produced by go). □ 



3.3 Dynamics of Event Graphs 

In the following, we will consider a timed event graph Q = (V, Q,£, M°,a). 
We further assume that there is at most one place between two transitions 
in an event graph. This assumption does not restrict the modeling power of 
event graphs. Now, if there is a place p between transitions i and j, this place 

clef 

is unique and can be denoted (ij), with initial marking Mfj = M°. The 

def 

maximal initial marking of all the places is denoted by m = max pe -p M°. 
3.3.1 State variables: the firing epochs 

In the Markovian case (all a q (n) have an exponential distribution), the mark- 
ing M(t) is a Markov process. The main problems with this approach are that 
the exponential distribution assumption may not be appropriate in many 
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cases 3 also the state space is potentially infinite (or very large) and the tra- 
jectory of the system is not easy to retrieve. 

In the following, it is merely assumed that the firing times are stationary 
processes. Here, the marking M(t) is no longer the appropriate state variable. 
Instead, let the functions {Xi(n),i £ Q,n = l, - } be the epoch of the 
beginning of the n-th firing of transition i. 

Note that the marking can be retrieved from the Xi(n) by the following 
formula: 



M p (t) = M p (0) + ^2l{X. p (n)+a. p (n) < t}-l{X p .(n)+a p .(n) < t} (3.5) 

71=1 

Transformation of the graph Any event graph can be transformed into an- 
other event graph with at most one token per place in its initial marking and 
with the same behavior. This is done by expanding places with more than 
one initial token into several places with one initial token as illustrated by 
Figure 3.3. 



to — 




Fig. 3.3. Expansion of a place with 3 initial tokens into three places with one 
initial token 



If G = (V, Q,£, M°,a) is an event graph with more than one token in 
place 7r, then 0' = (V , Q' ,£' , M° ,a') is a event graph where place it is 
replaced by M°(tt) — 1 dummy transitions and M°(tt) dummy places with 

clef 

one token initially. More precisely, let k = M°(tt) be the marking in place 
7r and i and j be the input and output transition of tt respectively. 

3 The Markovian approach can be extended to the case where the firing times 
are either exponential or deterministic [41]. This makes the marking M(t) semi- 
Markovian but the state space problem and the difficulty to study the trajectories 
are even more acute. 
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a' q (n), if g £ Q, 
0, if g G Q' \ Q, 



with the convention that qo = i and = j. 

By repeating the expansion for all places with more than one token ini- 
tially, one gets an event graph Q = (P, Q, £, M°,a) with a maximal marking 
to = 1 and such that the behavior of the initial graph is preserved. For each 
original transitions i £ Q and for all n, the firing epochs are the same under 
G and under Q: Xi{n) = Xi{n). 

In the following, we will assume that the initial marking in each place is 
at most one. 

3.3.2 The (max, plus) Semi-ring 

Definition 11. The (max,+) semi-ring is the set lU {— oo} equipped with 

,i , ■ , , , • def , def 

the two internal operations @ = max and <2> = +■ 

The neutral element for operation ® is — oo, which plays the role of 0. 
The unit element for operation ® is which plays the role of 1. Operation ® 
distributes with respect to ®. 

The main difference with the conventional ring (M, +, x) is the fact that 
the first operation ® is idempotent: a © a = a and does not have an inverse 
(hence the denomination semi- ring). 

The operations can be extended to matrices with the classical construc- 
tion. 

If A and B are both matrices of size n x to, then A © B is a matrix of 
size n x to with 



As for the product, if A and B are matrices of size nxf and I x to respectively, 
then A <8> B is a matrix of size n x to where 



(A © B)ij = Aij © Bij, i = 1 ■ ■ -n, j = 



1 • • • TO. 



(3.6) 



(A®B) itj = Q)A ik ®B kj , i = l--- n ,j = 1- m. (3.7) 
fc=i 
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Linear equations - Solving linear systems in (max,+) is rather different from 
the classical linear case. However, there exists one important class of linear 
systems where the solution exists and is unique. 

Lemma 15. A vectorial linear equation of the form X = A® X (Bb has a 

def 

minimal solution of the form X = A* ® 6 where A* = (B^1 A Z . This solution 
is unique if all the entries of A are finite. 

For a proof of this result, see for example [23]. An important special case 
is presented in the following corollary. 

Corollary 5. If matrix A is acyclic, then A* has all its entries finite and the 
vectorial linear equation X = A® X (Bb has a unique solution X = A* <g> b. 

3.3.3 Evolution Equation in the (max,+) semi-ring 

The autonomous case - Let us consider the case of a timed event graph 
= {'P, Q,£,M°,a) with no input transitions. 

We assume that this net is live and that the initial marking has at most 
one token per place. 

We also assume that the net satisfies a local FIFO assumption in each 
transition. Namely, the n-th firing to start is the n-th firing to finish. 

This local FIFO assumption can be enforced by constraining the firing 
times. For example if the firing time o~ q {n) of a transition q are non-decreasing 
in n, then q satisfies the FIFO assumption. However, this kind of assumptions 
on the firing time process are often too restrictive, since we want to deal with 
firing times forming stationary sequences. The local FIFO assumption can 
also be enforced by topological constraints. The easiest one is by recycling 
the transitions, as illustrated in Figure 3.4. 



In a recycled transition only one firing can take place at any given time, 
since the token in the recycled place is frozen and will only be available once 
the current firing is over. This enforces the FIFO assumption. 

Now, under these assumptions, and according to the firing rule, a transi- 
tion i in Q, starts its n-th firing at time 




Fig. 3.4. A recycled transition 
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Xi(n) d = -ai{l), Vn<0, (3.8) 
Xi(n) = max Xj(n- M^+a^n- M°J, \/n > 0. (3.9) 

Using the (max,+) notation, one gets 

Xi{n) = X 3 {n - M%) ® cj 3 {n - M%). (3.10) 
je-i 

This equation can be seen as a linear equation between the variables 
Xi(n), with coefficients <Jj(n — Mj'J. 

When written in vectorial form, it becomes 

X(n) = A(0,n)®X(n)®A(l,n)®X(n-l), Vn > 0. (3.11) 

where X(n) is the vector (Xi(n), ■ ■ ■ Xq(tj))* and for fc € {0, 1}, ^4(fc, n) 
is a Q x Q matrix defined by 

A(k,n^ d ^\^ n - k) if + f^ = fc ' (3.12) 
J [ —oo otherwise. 

Equation 3.11 is implicit but can be made explicit. Since Q is live, then 
all circuits are marked initially. This implies that matrix A(0,n) does not 
contain any circuit. Therefore, A(0,n)* is finite and the solution of Equation 
3.11 is unique, by applying corollary 5. 

X(n) = A(0,n)* ®A(l,n) ®X(n- l),Vn > 0. (3.13) 
def 

If one define A(n) = A(0, n)* ® A(l, n), then the standard form of the 
evolution equation of an autonomous event graph becomes: 

X(n)~(- a{1) Vn "°' (3 14) 

The non- autonomous case - Now, we consider an event graph which is live 
with at most one initial token per place and which contains transitions with 
no incoming places ( input transition). This case will also be referred to as 
the "open case" . Then the set of all the transitions Q is split into two parts: 
input transitions (set Qi of size Qj) and all other transitions, (set Qn of size 
Qn). 

Recall that firing of input transitions does not obey the standard firing 
rule since they have no input places. The sequence of firing epochs of input 
transition k is given beforehand and denoted Uk{n). 

For the dynamics of the other transitions (transitions in Qn), we define 
the matrix B of size Qn x Qk which gives the connections of the inputs with 
the other transitions. More precisely, the entry i, j in B is if there is a place 
(initially empty) between the input transition j and transition i, and is — oo 
otherwise. 
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The evolution equation of a non-autonomous event graph is then of the 
form 

X(n) = A(0, n) ® X(n) © ,4(1, n) ® X(n - 1) © B ® U(n), Vrc > 0. (3.15) 

Where X(n) is a vector of size Qn and A(0,n) as well as A(l, n) are 
matrices of size Qn x Qn and are defined as in the autonomous case (see 
Equation 3.12). 

Since the graph is live, it does not contain any empty circuit and matrix 
A(0,n)* exists and is finite. By using Lemma 15 once again, the unique 
solution of Equation 3.15 is for all n > 0, 

X(n) = A(0, nf ® A(l, n) ® X(n - 1) © A{0, n)*B ® U(n). (3.16) 
dpf 

Let A(n) = A{0, n)* ® A{1, n) and B(n) = A{0, n)*B. 

Then, the standard evolution equation in the open case becomes 

X{n) = A{n) ® X(n - 1) © B(n) ® U(n). (3.17) 

In this framework, whenever we refer to an open (max, plus) linear system, 
we refer to a discrete event system for which the evolution equation can be 
written under the form of Equation 3.17. Most of the time, these systems can 
be modeled by (stochastic) event graphs as we just did. The following section 
presents several examples of such systems. 

3.4 Queuing networks 

The aim of this section is to give some practical examples of systems 
from queuing, manufacturing and communication that fall in the class of 
(max, plus) linear systems. For each case we will exhibit the stochastic event 
graphs model as well as the corresponding (max, plus) standard evolution 
equation. 

3.4.1 The G/G/l queue 

This first example that is going to be detained is the G/G/l queue. 

Figure 3.5 shows the event graph model of a G/G/l queue. This event 
graph has two transitions, (one input transition modeling the arrivals, and 
one transition modeling the service) and two places (one for the infinite buffer 
and one to impose the mono-server semantics which also imposes the FIFO 
assumption). Note that tokens have different meanings according to the place 
where they are in. A token in place p\ represents a customer (a frozen token 
in place p\ is a customer being served, an available token in place p\ is a 
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Fig. 3.5. Event graph model of a G/G/l queue 



customer waiting for service). However the token in place P2 represents the 
server (if frozen, the server is busy, if not, the server is idle). 

Should the initial marking in the recycling place P2 be k, this becomes 
a G/G/k queue. If the recycling place is removed, this becomes a G/G/oo 
queue. Note that only the G/G/l queue is (max,+) linear for general firing 
times, since it is the only one complying with the FIFO assumption. 

For the G/G/l queue, the evolution equation is given in Equation 3.18: 



— X\{n) is the epoch of the n-firing time of transition 1 {beginning of the 
n-th service), 

— U(n) is the the epoch of the n-firing time of transition (arrival time of 
the n-th customer), 

— A(n) = A(l, n) is a 1 x 1 matrix equal to the n-th firing time of transition 
1 (duration of the n — 1-th service), A(n) = <ri(n — 1), 

— B(n) is a 1 x 1 matrix which gives the traveling time of a token from its 
entry in the system till reaching transition 1: B(n) = 0. 

Since the equation is scalar in this case, it can be written using classical 
notation: 



which is classical in queuing theory. 

The advantage of the (max,+) notation will better appear in the following 
example where a network of queues is considered. 

3.4.2 Queues in tandem 

Consider a network of Q G/G/l queues in tandem, with all queues initially 
empty. 

The event graph model of the network of queues has one input transition 
and Q ordinary transitions modeling respectively the arrival of the customers 



Xi(n) = A(n) ® Xi(n - 1) © B(n) ® U(n) 



(3.18) 



where 



Xi(n) = max(o-i(n - 1) + Xi(n - 1), U(n)) 
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Fig. 3.6. Event graph model of queues in tandem 



and the Q servers. An example of the event graph model of queues in tandem 
is given in Figure 3.6. 
In this case, we get 



A(0,n) = 



/ —00 


— 00 • 


— 00 


— 00 


— oo\ 


ai (n) 


— 00 • 


— 00 


— 00 


— 00 


— 00 


<7 2 (rc) • 


— 00 


— 00 


— 00 


— 00 


— 00 • 


• o-Q-i(n) 


— 00 


— 00 


\ —00 


— 00 • 


— 00 


u Q {n) 


— 00 J 



( o\{n — 1) —oo 
—oo o-2.{n — 1) 



A(l,n) 



-oo 

-00 



— 00 
\ —00 



— 00 

— 00 



• <TQ-i(n- 1) 



-00 \ 
-00 

-00 



-oo o-q(tj-I)/ 



/ o \ 

-00 
-00 



B = 



— 00 

V-oo/ 

The matrix A(0,n) is clearly acyclic and applying Corollary 5, the matrix 
A(0,n)* exists. Therefore, the matrix A(n) = A(0,n)* ® j4(1,tj) is given by : 



~°P_ 1 if * < 3, (3 1Q) 

E*=j a k (n) + aj (n - 1) if i > j. 



and the vector B{n) = A(0, n)* <2> B is given by 
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B(n) = 





oi (n) 
0i (n) +0 2 (n) 



\ 



The firing instants of the transitions 1 to Q , X(n) = (Xi(n), ■ ■ ■ Xq(ti)) 
corresponds to the start of the service times in the queues. The exogenous 
arrival instants are denoted U(n) and the evolution equation is X(n) = 
A{n) ® X(n - 1) © B{n) ® [/(«) for all n > 1. 

3.4.3 Kanban systems 

Consider a system of single server queues in tandem as depicted in Figure 
3.7. 




Fig. 3.7. Kanban system with blocking before service 



The first queue is fed by the external arrivals and has an infinite capacity 
buffer. All the other queues have a finite capacity buffer. The mechanism for 
service is blocking before service: A customer can start its service at queue 
i if the buffer as queue i + 1 is not full. Otherwise, it stays in buffer i until 
some customer leaves queue i + 1 so that it can start its service. 

In Figure 3.7, the places of type p\ impose the mono-server semantics as 
previously, places of type P2 represent the buffers in front of each server and 
places of type pz enforce the finite capacity of the buffers. The initial number 
of tokens in a place p% is the buffer size for that queue (which is one in the 
figure). The fact that the total number of tokens remains constant in the 
circuit made of places pi and ps imposes the finite buffer and the blocking. 
When the buffer is full, place p% is empty and no service can start in the 
previous queue. As soon as a customer leaves the queue, one token is released 
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in place pz and the previous queue can start one service, should a customer 
be waiting for service there. 

The (max,+) equation of that system is again of dimension Q, of the usual 
form 

X(n) = A{n) ® X(n - 1) © B(n) ® U(n). 

In that case, we get the same matrix A(0,n) as for tandem queues with 
infinite buffers (see Section 3.4.2). 



A(0,n) = 



( —oo —oo • 
01(71) —oo •• 
-oo 02(71) ■ ■ 



-00 
-00 
-00 



-00 — 00 \ 
-00 —oo 
-00 —oo 



—oo —oo ••• (jQ-i{n) —oo —oo 
\ —oo —oo ••• —oo 0q(ti) — oo y 

However, the matrix A(l,n) is different: 



/ 01(71 — 1) 02(71 — 1) —00 
—00 02(71 — 1) 03(71 — 1) 



A(l,n) = 



-00 
-00 



-oo\ 
-00 



—00 
—00 



— 00 

— 00 



0q-i(ti- 1) 0q(ti - 1) 

-00 0q(71-1) 



/ \ 

-00 
-00 



B = 



—00 
V-oo/ 

Therefore, the matrix A(n) = A(0,n)* ® A(l,n) is given by 

( —00 if i < j — 1, 

(A(n))ij = ^ (?j(n- 1) if i = j" — 1, 

I El=i + 0j(7i - 1) if i > j. 

and the matrix B(n) = A(0, n)* ® -B is given by 



S(n) = 





01 (ti) 
01(71) +02(71) 



\ 



V E?=>*(») / 



(3.20) 
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One can also analyze tandem queue with manufacturing blocking (block- 
ing after service), which can be modeled by event graphs as well. The main 
characteristic needed to keep the (max,+) linearity is that no customer is 
ever lost and the preservation of the FIFO property. 

3.4.4 Window flow control 

This example is taken from communication networks. If a node Sender sends 
information to Receiver through a packet switched network made of Q nodes 
(counting Receiver but not Sender ), a flow control is a mechanism which 
slows Sender in order to avoid overloads. A simple mechanism which is used 
by many protocols (such as TCP) is the window flow control. This is used 
to limit the total number of packets in the network. At reception of the n-th 
packet, Receiver sends an acknowledgment to Sender. As for Sender, it sends 
its n-th packet only if the acknowledgment for packet number n — W has 
already been received. The parameter W is called the window size. 



Sender Pi 1 2 




Fig. 3.8. Window flow control model 



The event graph model of a simple window flow control mechanism is 
displayed in Figure 3.8. Transition Sender is an input transition and models 
the sender. Transition Receiver models the receiver. The transitions from 
Sender to Receiver are the nodes (with a mono-server semantic) forming the 
route in the network for the packets from Sender to Receiver. The transitions 
from Receiver to Sender model the nodes crossed by the acknowledgments. 
Tokens in place pi represent the packets that a has produced which are 
waiting for a new acknowledgment to come to be sent. Tokens in place P2 
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represent the number of acknowledgments that have been received and which 
allow sending new packets in the network. When the place P2 is empty, W 
packets have been sent without any reception of acknowledgments and the 
next packet to be sent has to wait before heading to the network. The initial 
number of tokens in the whole circuit is W and remains constant. To ease the 
matrix representation, the initial marking is such that each place (starting 
with place P2 and going backward) contains at most one token. If the total 
number of nodes in the circuit is smaller than W, this is done by adding 
"dummy" nodes, with no service time, just as described in 3.3. 
Since W > 0, the matrix A(0,n) is acyclic (lower diagonal). 



A(0,n) = 



1 —00 


— 00 • 


— 00 


— 00 • • 


• — 00 \ 


ffi (n) 


— 00 • 


— 00 


— 00 • • 


• —00 


— 00 


02(71) • 


— 00 


— 00 • • 


• —00 


— 00 


— 00 • 


• VQ-w{n) 


— 00 • • 


• —00 


— 00 


— 00 • 


— 00 


— 00 • • 


• —00 


\ —00 


— 00 • 


— 00 


— 00 • • 


• —00 / 



The matrix A{l,n) captures the recycling places with one initial token 
and all those W places containing one initial token. We have 



4(1,71 + 1) = 



f (7i (n) —oo 
-oo 02(71) 



— 00 —00 

— 00 —00 



-00 
-00 



-00 
-00 



—00 —00 
&Q-w+i(n) -00 

VQ-w+i{n) UQ-w+2{n) 



(r Q {n)\ 
—00 



—00 
—00 



\ —00 —00 
and the vector B is 



0-q_i(ti) a Q (n)J 



( \ 

-00 
-00 



B = 



—00 
V-oo/ 

The matrix A(n) = A(0, n)* ® A(l,n) is given by 
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' El=i °k(n) +a j {n-l) if 1 < j < i < Q - W + 1, 



E*=i + - 1) if 1 < * < <2 - w + 1 and i = Q, 

<jj{n -1) \fi>Q-W + l and j < i < j + 1, 

— oo otherwise. 

(3.21) 

and the vector B(n) = A(0, n)* ® B is given by 



fl(n). 



_ / E/t=i 

\ —00 



\u k {n) ifl<t<Q-W + l, 
otherwise. 



(3.22) 



Finally, the standard evolution equation of the system is again X{n) = 
A(n) ® X(n - 1) © B(n) ® U(n), with ^(n) and B(n) as above and U(n) 
being the time when the n-th packet is emitted by Sender. 

3.4.5 Leaky buckets 

This example is also taken from communication networks. In large networks, 
leaky buckets are a commun device often used to cut off bursts in the entry 
traffic. They have been extensively studied, on a stochastic [111] as well as 
deterministic point of view [39, 110]. 



paquet 

infinite buffer entering 
Input paqucts ^ the net work 



token 
buffer 



network 



token 
generation 



Fig. 3.9. A Leaky bucket 



A leaky bucket is represented in Figure 3.9. The input stream is buffered 
in an infinite buffer and each paquet has to get a token in order to enter the 
network. Tokens are generated according to a process with intensity p and 
are kept in a buffer with capacity a. Therefore arrival bursts of size larger 
than a cannot enter the network and are filtered by the leaky bucket. 
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Fig. 3.10. Event graph model of a leaky bucket 



If we assume that the token generation is done according to a deterministic 
process which lets p tokens enter the token buffer every unit of time, then a 
leaky bucket can be modeled by an event graph, represented in Figure 3.10. 

Again, one can construct matrices A(n) and B(n) such that the standard 
evolution equation of the system is X(n) = A(n) ® X(n — 1) © B(n) ® U(n), 
with U(n) being the time when the n-th packet arrives in the leaky bucket 
and X a (n) being the time when it enters the network. 

3.5 Lindley's equation for (max,+) systems 

In this section, we will show how the evolution equation on the firing epochs 
can help to establish an equation for the response time of a (max, plus) system. 
By analogy with the G/G/l case, this equation will be called the Lindley's 
equation. 

First, let us state the assumptions that are made on the system under 
study. We consider an open stochastic event graph G, which satisfies all the 
assumption which as necessary to have a linear evolution equation of the 
form 3.17, i.e. it is live and locally FIFO. Here, we further assume that Q 
have only one input transition (denoted qo) and is input-connected. We also 
assume that the event graph is initially in an input deadlock: no transition 
can fire (except the input transition). These last assumptions imply that for 
every transition q in Q, there exists a path from qo to q that contains no 
tokens initially. More formally, this can be written using 3.4 as, 



Note however, that the system being in an initial input deadlock does not 
mean that the initial marking is zero in all the places. For instance, the 
network depicted in Figure 3.7 is in an initial deadlock, since 1/(1) = 1/(2) = 
• • • = L(Q) = 0. However, some places as for example the recycling places or 
the places of type p3 have a positive initial marking. 



L(q) = 0,Vge S. 
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We recall that X q (n) is the epoch when transition q starts to fire for the 
n-th time. Note that because of Lemma 14 and using the fact that L(q) = 0, 
X q (n) is also the epoch when the n-th token generated by the input is fired 
by transition q. 

def 

Now, let W n be a vector , with its q-th component equal to: W% = 
X q (n) — U(n). Using Lemma 14 again, W% can be seen as the traveling time 
for customer n between its entrance in the system and its passage in transition 

Using the standard evolution equation for X(n), namely 

X(n + 1) = A(n) ® X(n) © B{n) ® U(n + 1), 

and if we consider only the component Xi(n) and the row i in matrix A(n) 
(denoted Ai(n)), we can subtract U(n + 1) (a scalar since we have a single 
input) on each side of this equation and we get: 

Wl +1 = Ai{n) ® X(n) © B(n)i ® U(n + 1) - U(n + 1). 

We can rewrite this as 

W l n+1 = Ai{n) ® (X(n) - U{n + 1)) © B{n)i 

= {Ai{n) - t„) ® (X(n) - U(n)) © B(n) i; 

withT n d =U{n + l)-U{n) 

If we write this last equality for W n+ i in vectorial form, we get 

W n+1 = A(n) ® £'(-r 7i ) ® W n © B(n), 

where -D(/i) is the diagonal matrix with h on the diagonal and — oo ev- 
erywhere else. 

This recursion is a generalization of the Lindley's equation in the case of 
a network. This equation was also derived in [25] . 

By using elementary matrix operations in the (max,+) algebra, the equa- 
tion can also be developed into: 

n 

W n+ i=B(n)e0Ci, (3.23) 
i=i 

with 

n 

C t = (g) (A(j) ® D{- Tj )) ®B{n-i- 1), 
def 

where we define for convenience B(0) = (— oo, • • • , — oo)*. 



Part II 



Admission and routing control 
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This part shows how the general theorems presented in Part I can be 
applied in networks of queues or Petri nets which form timed event graphs. 

The first two chapters focus on the problem of admission control where 
the controller is positioned at the entrance of the network and decides 
-either to accept (packets are allowed in the network and continue their routes 
to the destination) 

- or to reject the packet ( in which case, the packet is lost forever). 

Chapter 4 gives the form of the optimal admission control in open loop 
for a stochastic event graph. 

Chapter 5 discusses the applicability of this technique in realistic telecom- 
munication networks. 

The next chapter (6) considers the routing control problem where the 
controller decides the route that the packet must follow rather than it should 
accept or reject the incoming packet. The results in the general case are 
much weaker than for the admission problem. The optimal policy is shown to 
have rates for each system when a balanced policy is admissible. However the 
computation of the optimal rates is not done in the general case. Even when 
the optimal rates are known, the optimal policy remains difficult to find. 

Only special cases are solved in the following. The case where the parallel 
systems have the same service distributions is solved completely in Chapter 
6. The case of two deterministic systems in parallel is solved in Chapter 7. 

These examples illustrate the intrinsic difficulty of the computation of 
the optimal rates as well as the determination of the optimal policy in the 
general case. It seems doubtful that a general construction, merely based on 
the parameters of the system can always be given in closed form. 
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4.1 Introduction 

The work on admission control in queuing systems can be split into two main 
domains, depending on the the information available for control decisions. 
When some knowledge on the state is available (closed loop systems), then the 
optimal control policy is usually based on dynamic programming techniques 
[52, 95] and more recently in [100]. When no state information is available 
(open loop control), then the control is often transformed into a problem of 
assigning an optimal constant input rate, see Section II in [104]. Instead of a 
dynamic control problem we are then faced with an optimization of a single 
parameter. This presentation falls in between since it belongs to the open 
loop framework but remains dynamic in some sense since control decisions 
will be taken for each arriving customer. 

The general result which is proved in this chapter is that when the admis- 
sion rate is fixed, then the individual arrivals have to be distributed evenly 
over time in order to minimize the average waiting time (or the workload). 
The property of the cost function used here in order to prove the optimization 
result is multimodularity. 

We focus on systems with the following properties which can be consider 
as rather realistic for classiacl networks models (more on this in Chapter 5). 

1. We consider a queuing network with one input node. This network is 
assumed to be a stochastic event graph (queues in tandem for example, 
fit in this framework, see Section 3.3). 

2. The service times of the customers admitted in the network may be any 
stationary process. 

3. We look at performance measures such as the average workload or the 
average traveling time. More generally, any performance measure which 
is a weighted average of expectations of convex functions of the workload 
(or traveling time) to any of the queues can be considered, (see Lemmas 
22 and 5 for a precise formulation). 

In this framework, 

we show that the following result holds: under all admission policies, with 
an asymptotic fraction p of acceptance, the balanced policy with rate p is 
optimal. This policy can be given by: 
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a n = [np\ - [(n - l)p\, 

where [X\ denotes the largest integer smaller or equal to X, and a n = 
1 (0) if the 7i-th arrival is accepted (rejected). The intuitive idea behind it 
is that for stationary arrivals, balanced admissions spread out the entrance 
times of the admitted jobs "most equally" . 

The main objective of the chapter is to verify the conditions of Lemma 
22 (or Lemma 5). The main point is the verification of the multimodularity 
property, its proof for the single server case relies on Lindley's equation. For 
the stochastic event graph, we use the vectorial Lindley's equation established 
in Chapter 3. Although technically and notationally more involved, it turns 
out that the verification of multimodularity in this general setting has the 
same structure as for the one-server queue with a general service distribution 
and "first in first out" service discipline. Via a counter example, we point out 
in Section 4.2.1 that the multimodularity property does not hold sample path 
wise (when we consider a deterministic sequence of service times which are 
not equal). Therefore, we consider the expected workloads (expected traveling 
times) and we apply a coupling of the service times (inter-arrival times). See 
Section 4.3 for details. 

4.1.1 Organization of the chapter 

The chapter is structured as follows. Section 4.2 introduces the definitions and 
Section 4.3 studies the particular case of a FIFO (First In First Out) queue, 
to give an idea of the proof in the general case. The main goal of Section 
4.4 is to show that the workload in a (max,+) linear system is multimodular 
with respect to the arrival sequence. In Section 4.5, we give the proof of 
multimodularity for the traveling time of a customer in the system. Finally, 
in Section 4.6, we show that a balanced admission policy is optimal among all 
open-loop policies with a given admission rate by applying our results from 
Chapter 1. 

4.2 Multimodularity and admission control 
4.2.1 Admission policy: the time slot approach 

In the rest of the chapter, we will use the following notations: 

let {T;}; g n be the instants of arrival opportunities, with the convention 

that Ti = 0. Since all the rest of the notations are based on the original 

sequence, this can be considered as a time driven approach. 

We denote by 6i the ith interval length, that is: 6i = T; +i — T;. We assume 

that <$o = 0. From now, the sequence T is fixed (and hence the sequence 6). 
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As for the customers, they arrive by batches. Their arrival is defined 
through an arrival sequence which is a sequence of integer numbers, a = 
(ai, a>2, ■ ■ ■ , a>N, ■ ■ ■ ), where gives the number of customers entering to the 
queue at time Tj. For convenience, we introduce the value oo = 0. In Sec- 
tion 4.6 the sequence a will be seen as a control sequence over the arrivals 
but will be used in the same way. 

Furthermore, each individual customer carries a load. The load of the 
j — th customer is denoted a, . 

The counting function n{i) is the number of individual arrivals by time 

T • 

The function v{i) is the number of time-intervals elapsed when the ith 
individual customer enters the system. We assume that ^(0) = and 

def m 
v{i) = minjm : > i}- (4.2) 

3=1 

The function Ti(a) is the time elapsed between the i-th and (i + l)-th 
individual arrivals, defined by: 

v(i+l)-l 

n(a) d = ^, = T V(Z+1) -T v(i) i < k(N), and (4.3) 

j=v{i) 

Figure 4.1 illustrates all these preliminary definitions. 



4.3 The FIFO queue 

In this section, we assume that the arrival stream enters a single FIFO queue. 
This is a simple case and the proofs presented here are typical of what hap- 
pens in the general framework. The system considered is a G/G/l queue with 
batch arrivals. Note that according to Section 3.4.1, this system is (max,+) 
linear. 

Here, if a; = k, then this means that k customers enter the queue at 
time Tfc. We denote by Wj(a) , the workload in the system at the arrival 
time of the j-th individual arrival, T„^ K ^, and caused by arrivals up to and 
including the (j — l)-th individual arrival, for j < k(N). Also, W K (jv) + i(a) 
is the workload just after time Tjv under the arrival sequence (a±, ■ ■ ■ , ajv). 
The function VVj (a) satisfies the following recurrence equation. 



82 



4 Admission control in stochastic event graphs 



number of arrivals 



TO 



T3 



T2 



T6 



r 5 = 



TV 



Ti 
a : 



T 2 T 3 
1 



T 4 
2 



T 5 T 6 T 7 
1 

—d 6 



2 



Tg Tio 

1 







Fig. 4.1. an example 



\N j+1 (a) = (\N j (a) + a j -T j )W0 j < k(N). (4.4) 

The solution of this equation is given by the expanded Lindley's equation. 
For W K( jv )+1 (a), we get 

0, max (o-j - r,) J . (4.5) 
def 

For convenience, we denote Wn{cl) = W K (jv)+i( a ) and 

d f K ^ 

Using this definition, we have: 

W N (a) =max (0, w K ( N ), ■ ■ ■ ,wi) . 



4.3.1 Coupling of the service times with the customers 

The rest of the section is devoted to proving multimodularity of the workload. 
Unfortunately, this does not hold on sample paths (as illustrated by the 
following example). We have to use a coupling of the service times with the 
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customers entering the queue, and then, by assuming that the service times 
are stationary, we will prove multimodularity of the expected workload. 

First, we will illustrate the difficulty of attaching the service times to 
customers while insuring that multimodularity holds, through an example. 

An example We consider a single queue with a sequence of service times 
a(n) = (4, 1, 1, 1, 1, 1, • • • ) and with the integer points as arrival epochs. 

We focus on the workload immediately after time T$ = 7 under the arrival 
streams: 

a= (0,0, 1,0,0,1,1,0) (4.6) 
a + ei = (1,0,1,0,0,1,1,0) (4.7) 
a - s 7 = (0,0, 1,0,0,0,2,0) (4.8) 
a - s 7 + ei = (1,0, 1,0,0,0,2,0) (4.9) 

The workload satisfies: W s {a) = 1, W s (a + ei) = 0, W s {a + e\ — s 7 ) = 
1, W 8 (a — s 7 ) = 1, as shown in Figure 4.2. 




W(a) = 1 





W{a - d ) = W(a - d - d 6 ) = 1 

Fig. 4.2. The workload is not multimodular. 



This shows that the function Wg is not multimodular, since Wg(a) + 
W 8 (a + ei - s 7 ) > W 8 (a + ei) + W 8 (a - s 7 ). 

However, under a proper coupling of the service times with the customers, 
then W can be made multimodular. 

We couple the service times with the customers entering the queue in the 
following way. 
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a = 1 1 1 

<T = <Ti <T2 0"3 

a + ei =101001 1 

a = (To 0"! <T2 0"3 

a-s 7 =001000 2 

C = Ci <T 2 ,<T3 

o + ei-s 7 = 101000 2 

a = (To O"! <T2, <T3 

Under this coupling c, the workloads become as in Figure 4.3. We see that 
W£{a) = 0, Wg(a + e x ) = 0, Wg{a + e x - s 7 ) = 1, VF 8 c (o - s 7 ) = 1, and VF 8 C 
satisfies the multimodular inequality (which is an equality here), 



J\ N\. 



W(a) = 





V7(a-d ) = VP(a - d - d 6 ) = 1 

Fig. 4.3. Multimodularity of the workload under proper coupling. 



General coupling In general, the coupling of the service times with the arrival 
stream is done in the following way. 

— Let o be an arbitrary arrival sequence in N^: a = (01,02, • • • , ajv)- The 
service times are coupled with the customers entering the queue in the 
following fashion: With arriving batch a*, we attach the service times: 

a K(i-l) + l, f«(i-l)+2j " " " , a K(i)- 
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— With arrival sequence, a + ei, the service time sequence for batch <n,i ^ 1, 
is not modified and the batch attached with a± becomes: 

00, 01, 02, • • • , 0"«(1), 

where oo is a new service time. 

— If a — Sj + i e N^, 1 < j < N, the service time sequence for batch a>i,i ^ 
j, i ^ j + 1 , is not modified and the batch attached with a,- (which is not 
empty) becomes: 

and the batch attached with a>j + i becomes: 

In other words, the service time <J K (j) is moved from the j-th batch to the 
j + 1-th batch. 

— If a — ejv £ N N , the service time sequence for batch a>i,i ^ JV, is not 
modified and the batch attached with ajv (which was not empty) becomes: 

0' K (JV-1) + 1, 0' k (JV-1)+2, • • • , 0' K (JV)-1- 

The last service time <j k (n) has been removed. 

— If a + u+ ve N N , u, v € J 7 , u ^ v, then the coupling of the service times 
is obtained by composing the modification of the coupling associated with 
a induced by u and v. This construction is commutative. 

Note that this coupling strongly depends on the initial choice of a. If one 
changes the starting point a, then other service times will be chosen. Also, 
the coupling is not defined on o's which do not lay in the positive orthant, 

4.3.2 Multimodularity 

We choose a point a in and we construct the associated coupling c. We 
will denote by W^(.) the workload immediately after time Tjv under this 
coupling. 

Now, using the notations given in the preliminaries, we have the following 
results. 

Property 1. Let < i < N. If ai > 0, r re(i )_i(a - s i+ i) = r K(i )_i(a) + 6 i+ i, 
T K(i)( a — s i+i) = r K(i){d) — Si. All other Tj are unchanged. 

Proof. The addition of — Si + i corresponds to delaying the last acceptance at 
time Ti to time Tj+i. t k ^ is the time interval between Ti and the arrival 
instant of the next customer that arrives after time Ti. Hence, delaying the 
arrival from Ti to time Tj+i results in increasing r K ^)_i by Tj+i — Ti = Si. 
T K (j) decreases by that value. See Figure 4.1 for an illustration of this proof. 

□ 
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Property 2. Let < i < N. If a; > 0, w K ^(a — s^+i) = w K ^ + Si. All others 
Wj are left unchanged. 

Proof. Follows from Property 1 and the definition of Wj. rj 

Under the previous coupling, we have the following results. 
Lemma 16. Let h : M — > M 6e a nondecreasing convex function. Then 
h o W^(o) + /i o V7^(o + tt + i))</io W^(o + «) + /io W^,(a + u), 
/or oii «,ue J and a swc/i i/iai a + u + v, a + u and a + v are in N N . 
Proof. 

— Let us first consider the case < i, j < N and a>i > 0, a>j > 0. In this case, 
using the previous Property 2, we have: 

W%,(a - s i+1 ) = max(W^(a),w K{i) +6i) (4.10) 
W c N {a-s j+1 ) = max(W^(a),w K(i) + 6j) (4.11) 
Wx(a-s i+ i -s j+ i) = max(W^(a), +Si,w K ^) +Sj). (4.12) 

Therefore, we have 

max(W^(a), W^(a - s i+1 - s j+1 )) = max(W^(a - s i+1 ), W^(a - s j+1 )), 
W c N {a) + W c N (a - s i+1 - s j+1 j) < W c N {a - s i+1 ) + W c N (a - s j+1 \A.U) 

This means that the two dimensional vector (W^(a), W^(a — s,+i —Sj + i)) 
is weakly majorized by (W^(a — Sj+i), W^(a — Sj + i)). By using Theorem 
B.2, p. 109 in [88], for any non-decreasing convex function h, h(W^(a)) + 
h(W c N {a-s i+1 -s j+1 )) <h{W^{a-s i+1 ))+h{W c N {a-s J+1 )). 

— Let us now assume that i = 0. This case corresponds to the arrival of an 
extra customer at time Ti = 0. This customer brings a load that we denote 
oo. In this case, we have: 

W^ia + ex) = max(VF^(o),wi +(<t -t )). (4.14) 

This case is treated similarly to the case above since (a + e± ) is of the 
form max(Wjy(a), X), where X = w\ + (<ro — to)). 

— If i = N and ajv > 0, then we have: 

W c N (a-e N ) = (W c N (a)-a K{N) ) + . (4.15) 
Here, for some X (see the first two cases) 

Wff(a-v) = max(WS,(a),X), (4.16) 



4.3 The FIFO queue 87 

and 

W c N {a-v-e N ) = (max(^(a), X) - a <N) )+. (4.17) 

By a case analysis, we see that if W^(a — v — ejy) = then, W^(a — ejy) = 
and by monotonicity of ft, 

h o W^(o - ejy) + ho W^(a -v)>h° W^{a) + ho W^(a -v- e N ). 

If W^(a -v- ejy) > 0, then, 

W c N {a -v)- W c N {a - v - e N ) = a K ( N ). 

This yields, 

W c N {a) + W c N {a -v-e N )< W c N {a - e N ) + W c N {a - v). 

On the other hand, it is direct to see that 

max(W^(o), W^(a - v - e N )) < m&x(W^(a - e N ) + W^(a - v)). 

Again, by using the same Schur convexity property, for h convex and in- 
creasing, 

h o Wn(ci) +hoWfj(a - v - e N ) < h o W^(a - e N ) +hoW^(a - v). 

This concludes the proof. rj 

Theorem 17. Suppose {a n }^ =1 is a stationary sequence, then the function 
Ea(ho Wn) is multimodular, where 1^ denotes the expectation w.r.t. the 
sequence {a n } . 

Proof. Let o be an arbitrary point in . Construct the associated coupling 
c of the services times. Under this coupling and for all i, j, Lemma 16 shows 
that 

h o W c N {a + u) + ho W c N {a + v) - h o W c N {a) - h o W c N {a + u + v) > 0, 
u ^ v, u,v £ T . Therefore, 

E a {hoW c N {a+u)+hoW c N {a+v)-hoW c N {a)-hoW c N {a+u+v) S j >0. 

By the stationary assumption on the service times and using the fact that 
under coupling c, the service times involved in W^(.) are always consecutive, 

Kh°W c N (.)=E a h°W N (.), 

for the points a + u, a + v, a + u + v and a since the expectation is invariant 
with respect to the shift operator. Finally we get 

E CT (h(W N (a + u)) + h(W N (a + v))-h(W N (a))-h(W N (a + u + v))j > 0. 

□ 
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4.4 (max,+) systems with one input: multimodularity 

This section will generalize the multimodularity properties to the case of an 
arbitrary network which is (max,+) linear, has one input, is connected to its 
input and initially input-deadlocked. 

The main result established in this section is that the expectation of W% 
is multimodular for all transition q. The proof is very similar to the case of 
the single queue and is made, surprisingly, even easier by using the vectorial 
form of the Lindley equation in the (max,+) algebra. 

In the following, we will also often use the following transformation for 
notational convenience. If X is a vector of size Q, then [X] is a diagonal 
matrix of size Q x Q, with the vector X on the diagonal and — oo elsewhere. 

The multidimensional coupling of the service times in each transition with 
the arriving customers is done similarly as in the one queue case. 

We construct a coupling for transition q which is independent of the cou- 
pling for any other transition. 

Let a be an arbitrary arrival sequence in : a = (oi, 02, • • • , ajv)- The 
service times are coupled with the customers entering the system in the fol- 
lowing fashion: 

— With arriving batch a; (if size we attach the respective service times: 

q q q 

+ 1 ' °k(i-1)+2 ' " " " ' • 

— With arrival sequence, a + e±, the service time sequence for batch a>i, i ^ 
is not modified and the service times attached to a± now becomes under 
this coupling: <7q, af, <rf , • • • , v q K ^ where <7g is a new service time. 

— If a— Sj + i e , then with arrival sequence, a—Sj + i, 1 < j < N the service 
time sequence for batch a>i,i ^ j,j — 1 is not modified, the service times 
attached to a>j-i become: CT^ j _ 2 ^ +1 ,a^ j _ 2 ^ +2 , • • • , and the ser- 
vice times attached to batch a, also become: ■ ■ ■ i "*^)- 
In other words, the service time <J q K ^_^ is moved from the (j — l)-th batch 
to the j-th batch. 

— If a— ejv € N N , then with arrival sequence, a—eN, the service time sequence 
for batch a>i,i ^ N is not modified and the service times attached to o« 
(which is not empty) become: <J^ A ,_ 1)+1 , ct^ jv _ 1)+2 , • • • ,o"^ iv) _ 1 . The last 
service time c*(jv) has been removed. 

— If a + u + v £ N N , u, v e calF, then with arrival sequence, a + u + v, the 
coupling of the service times is obtained by composing the modification of 
the coupling associated with a induced by u and v. We note again that the 
construction is commutative. 

Now, using this coupling, we consider the traveling time vector of a poten- 
tial customer entering the system just after time Tjv under arriving stream a 
and its associated coupling. We denote this vector by Wjv(a) . This vector 
is defined by the following equation: 
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K (JV) 

Wjv(a) = B(k(N)) © d(a), (4.18) 
i=i 

with 

K (JV) 

Ci(a) = (^(j) ® 23(-Tj(a))) ® B(k(JV) - i - 1), (4.19) 

i=i 

where Tj(a) is defined in Equation 4.3. 

We have similar lemmas as in the one queue case, (see Properties 1-2). 

Lemma 17. Let < i < N. If a,-i > 0, then C K ^(a — Si+i) = D(6i) ® 
C K (,)(o). All other Cj are left unchanged. 

Proof. Using property 1, we have: 

£)(-r re(i )_i(a- s i+ i)) = D(-Si) £)(-r re(i )_i(a)) 

and 

D(-T K{i) (a- s i+1 )) = D(6i) ®-D(-r K(i) (a)) 
with all others Tj left unchanged. 

— Now, for every j > Cj(a — Si+i) does not involve D(—r K ^(a — 
or D(— r K (j)_i(a — Si+i)), and therefore is left unchanged. 

— If j < n(i) then Cj(a-s i+1 ) involves D(-T K ^(a-s i+1 )) and £>(-r K(i )_i(a- 
Si+i))- Since the matrices -D(x) commute with everything, and since 
,D(<5;) ® D(—6i) = E, the identity matrix, then, Cj(a — Si+i) is left un- 
changed. 

— Finally, if j = then Cj(a — Si+i) involves £?(— r K (j)(a — fii+i)) but not 
£?(— r K (j)_i(a — Si+i)). Using the fact that D(8i) commutes with all the 
other matrices, we have C K ^(a — Si+i) = D(6i) ® C K (j)(a). 

□ 

def 

In the following we will use, to simplify the equations, the matrices: Zi = 
D(6i)®C K{i) {a). 

Lemma 18. Let h = (hi, ■ ■ ■ , /iq) be such that for all q, h q : M — > M is an 

increasing convex function. Component-wise, we have for u, v £ ca/F, u ^ v, 

h q o W q N {a+u)®h q o W q N {a+v) >h q o W q N {a)®h q o w^(a+^+w). (4.20) 

Proof. First note as a general remark that for any matrix M and any positive 
number x, D(x) ® M © M = D(x) ® M. Now, as in the case of a single queue, 
we have to distinguish three cases. 
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— The case where < i, j < N and a>i,aj > 0. By commutativity of the ffi 
operator, we have 

W N (a-s i+1 ) = W N {a)®Z i+1 (4.21) 
W N (a - s j+1 ) = Wjv (a) © Z j+1 (4.22) 
Wjv (a - Sj+i - fli+i ) = W w (a) © Z j+1 ® Z i+1 . (4.23) 

Now using the distributivity of ffi w.r.t. ffi, we have [Wjv(a — Si+i)] ffi 
[Wjv(a - s j+ i)] = [W N (a)\ ffi [V7jv(a - - © [Z]j ffi [Z]*. This last 
equation interpreted in the classical algebra says that for each server q, the 
traveling time at time Tjv satisfies: 

W%(a - s t+1 )+W%(a- s j+1 ) > W%(a) + W^(a - s j+1 - s i+1 ). (4.24) 

The fact that h is increasing and convex component-wise, and using a case 
analysis with Equation (4.24) shows that 

h(W N (a - s i+1 )) + h(W q N (a - s j+1 )) 
> h(W N (a)) + h(W N (a - s j+1 - s i+1 )) 

— Now we examine the case where u = e±. As in the single queue case, this 
corresponds under our coupling to the arrival of an extra customer at time 
Ti = that has a service time that we denote <Tg in queue q. In this case, 
we have with n = k(N): 

n 

Wjv(o + ei) = Wjv(o) ffi (Aj ffi D{-Tj)) ffi fl^n). (4.25) 

3=0 

This case is treated as the general case, since Wjy(a + u) is of the form 
Wjv (a) ffi Z, for some vector 

, f » 

Z = (Aj ffi D(-Tj)) ffi Bi(n). 

3=0 

— If u = —eN and ajv > 0. This case corresponds to the removal of the last 
customer in the arrival batch, which happens to have arrived at time Tjv. 
In this case, we have: 

Wjv(o-eiv) = 5®Wiv(o)ffiO, 

where O is a vector composed of zeros and S is a diagonal matrix with 
^[9>9] = ~ a K(N)- Since we also have 

W N (a + v) =W N (a)®Z 

and 
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W N (a + v - ejy) = S ® (Wjv(a) © Z) © O = S ® Wjv(a) © 5 ® Z © O, 
def 

with Z = Zj, we get using distributivity of ® with respect to ffi, 

[VFjv(o-ejv)] ® [VMa + v)] 

= (S [VFjv(a)] ffi [O]) ® ([Wiv(a)] ffi [Z] 

= S®[W JV (a)]®[W JV (a)] 

ffi[0] ® [VFjv(o)] 

ffi5®[VFjv(a)]® [Z] 

ffi[0] ® [Z], 

and on the other hand, 

[VFjv(a)]®[VFjv(o + t;-ejv)] 

= [Wiv(a)] 5 [VFjv(o)] ffi [W N (a)} ® 5 ® [Z] ffi [VFjv(a)] ® [O]. 

Since all the matrices involved in these equations are diagonal, they com- 
mute and we have: 

[W N {a - ejy)] ® [VFjv(a + «)] = [VFjv(o)] ® [W N {a + v - e N )} ffi [O] ® [Z]. 

Since [O] ® [Z] is a non-negative diagonal matrix, the result is established 
by rewriting this equation in the conventional algebra. 

As with the function h, the proof is similar to the case of a single queue, rj 

Let a n ,n> 1 be the stochastic vectors with components <J^q e Q. 

Theorem 18. Suppose {a n } is a stationary sequence of stochastic vectors. 
Then, the function E (T /i(W^(o)) is multimodular, where 1^ denote the ex- 
pectation w.r.t. the service times in all the nodes of the system. 

Proof. The proof is similar to the one queue case. The coupling c is compatible 
with the shift of stochastic vectors of the service times in all nodes. Therefore, 
the inequality given in Equation (4.20) implies the multimodularity of the 
expected traveling time w.r.t. all service times. rj 

Note that this theorem proves that the traveling time for a customer 
arriving at time Tjv, that is, the time between its entrance in the system and 
its service in queue q, is multimodular, for all q and N. 

Also note that the case of event graphs with multiple or no entries as well 
as the case where the marked graph is not empty initially are intractable by 
this means. Indeed, the coupling of the service times to the customers is not 
feasible a priori in those cases. It will depend on the sequence of arrivals in 
the case of multiple entries, and on all the service times in the case of a closed 
system. 
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4.5 A dual policy: counting variable and waiting time 

In the previous section, we were interested in the study of the workload which 
is a criterion related to the server. Here we will focus on performances related 
to the customers, namely, the waiting time of the customers entering the 
network. This approach will be dual (in some particular sense) to the previous 
one and may be a more important issue for practical applications, where the 
customer satisfaction is more important than resource optimization. 

Previously, all quantities were indexed by n, the number of time slots. 
In this section, all quantities will rather be indexed by n(n), the number of 
arrivals. 

The counting sequence b, will be given in the following way: b n = v{n) — 
v{n - 1), n > 1 . For example if a = (0,0,1,0,0,2,0,1,1,0,0,1), then b = 
(3,0,2,1,3). Note that a and b represent the same information (up to the 
initial arrival); a n gives the number of arrivals at time slot n (this is a time 
driven concept), and the dual variable bk gives the number of time slots 
elapsed between the (fc — l)-th arrival and the k-th arrival (this is a event 
driven concept). 

4.5.1 Waiting time 

Let G be a (max,+) linear system with a single input satisfying the assump- 
tions given in Chapter 3. 

Then, the traveling time of the fc-th admitted customer to node i is de- 
noted by Wl(b). The vector Whip) satisfies the vectorial Lindley equation, 
using the function r defined in equation (4.3): 

W k +i(b) = A(k) ® D(-T k ) ® W k (b) e B(k), 
This can be written: 

n k 

Wk+i =B(fc)®0Ci, with d = Q$ (A(j) ® D(- Tj )) ® B(k - i - 1). 

i=l j=i 

This equation has essentially the same form as the equation (4.18). 

4.5.2 Coupling 

The coupling adapted in this case is essentially the dual of the coupling on the 
service times used previously. Here we rather couple the inter- arrival times, 
Si. This coupling uses the function v instead of the function k. 
We build the coupling d in the following way. 
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— Let b be an arbitrary arrival sequence in : b = (61,62,- ■■ , 6jv). The 
intervals are coupled with the customers entering the queue in the following 
fashion: With interval length bi, we attach the intervals: 

8 v (i-l) + l, <5i>(i-l)+2, ■ ■ ■ , 

— With b— ei, the interval sequence for length bi,i ^ 1 is not modified and the 
length attached with 61 (which was not empty) becomes: 61,62, - ■ ■ , ^(1), 
where £0 has been removed. 

— If b + Sj + i e , then with arrival sequence, b + Sj, 1 < j < N the interval 
sequence for length bi,i ^ j,j — 1 is not modified and the length attached 
with bj-i becomes: 

6„(j-2)+i,6 l/ (j-i) + 2, • • • , 8 v (j-i),8 v (j-i) + i 
and the length attached with bj (which is not empty) becomes: 

8 v (j-i)+2, ■ ■ ■ ,8 v (j). 

In other words, the interval 8 v (j_i) + i is moved from the j-th length to the 
(j — l)-th length. 

— If b + SN € , then with arrival sequence, 6 + sjv, the interval sequence for 
length bi,i ^ N is not modified and the length attached with 6jv becomes 

8 V (N-1) + 1, ^i/(JV-l)+2, • • • 1 ^i/(JV)-li ^i/(JV) + l- 

The last interval ^(jv)+i, is a new interval. 

— lib+u+v £ N N , then with arrival sequence, 6+ u+v, the coupling of the <5's 
is obtained by composing the modification of the coupling associated with 
b induced by u and v, u ^ i, u, v £ T. This construction is commutative. 

4.5.3 Multimodularity 

In this case, we will use the direct base T = {— ei, S2, ■ ■■ , ejv} rather than 
T = {ei, —S2, • • • , — ejv} used in the previous sections. 

Property 3. Let < i < N. If bi > 0, then r^(6 + Si+i) = r^(6) + 8 v ^ + i and 
Ti + i(6 + Si + i) = Ti + i(b) — 8 v (i) + i. All others tj are unchanged. 

Proof. By definition. See Figure 4.1 rj 

Lemma 19. LetO <i < N. Ifbi > 0, then 

C i+ i(b + s i+ i) = D{8 v{i]+ i) ® C i+ i(6). 

4H ot/ier are te/t unchanged. 

Proof. The proof is similar to the proof of Lemma 17 and follows from the 
Property 3 and the definition of Cj rj 
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In the following we will simplify the equations using the variables: 
Hpf 

Z^ 1 D{8 v{i)+1 )®C z+1 {b). 

Under the previous coupling, we have the following results: 

Lemma 20. Let h :~M9 -^>~M9 be a increasing convex function. Then, 

h o W(b) +ho W(b + u+v)<ho W(b + u) + ho W{bv), 

for all it, v £ T and all b such that b + u + v, b + u and b + v are in . 

Proof. The proof is essentially similar to the proof of Lemma 18. The proof 
follows from the following equalities by using the same technique as in the 
previous case. 

-If Ki,j<JV, 

W(6 + Si ) = W(b) e Zi-i (4.26) 

W(6 + Sj) = W{b) e Zj-i (4.27) 

W(b + S j + Si) = W{b) e Zj-i © Zi-i • (4.28) 

If u = — ei, 

W(b-e 1 )=W(b)®Z . (4.29) 

Ifu = e„, 

W{b + e N )=S®W{b)oO, (4.30) 
where O is a vector composed of zeros and S is a diagonal matrix with 

□ 

Theorem 19. 7/ tte intervals {8i}?l form a stationary sequence, then the 
function E$ (h o W) is multimodular. 

The proof is essentially similar to the proof of Theorem 18. 
4.6 Optimal admission sequence 

In this section, we study the admission control in a (max,+) linear system, 
as presented in Chapter 3. We use the multimodularity property to derive 
the optimality of the bracket admission sequence. 

We want to admit customers in a (max,+) linear system Q, under the 
constraint that in the long run, the fraction of customers lost be at most 
1 — p. 

Now the problem is to find which admission policy minimizes the trav- 
eling time to node i of a customer admitted in the system, as illustrated 
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arrivals 




\ os ^ (max,+) linear system 
Fig. 4.4. Illustration of the control problem 



in the figure 4.4. The admission is governed by a binary control sequence 
a = (ai, a>2, ■ ■ ■ ). If a n = 1 then the n-th customer is admitted to the queue. 
If a n = then the n-th customer is lost. 

Note that this control sequence can be seen as an arrival sequence as used 
in the previous sections. 

In this model, the vectors a n , n = 1, 2, ■ ■ ■ are service times of the cus- 
tomers which are admitted to the system, they correspond precisely to the 
definition of the firing times introduced in Section 3.3. There are no service 
times attached to the customers who are not admitted to the queue. This 
framework is natural in the stochastic Petri net context (see for example 
[20]). Note also that if the service times (or firing times) are independent, 
then this assumption is not needed and service times can as well be attached 
to all customers. 

4.6.1 Dual bracket sequences 

In this part, we claim that the arrival sequence a is bracket if and only the 
counting sequence b is also bracket up to an adequate choice of the phase. 
The following lemma uses results about bracket sequences which are detailed 
in Chapter 2. 

Lemma 21. If sequence a is a bracket sequence with rate p and phase 8, 
with 8 and p linearly independent over the rationals: 

a n = \p(n + 1) + 8\ - \jm + 8\ , 

then b is also bracket with rate 1/p and phase <f> = —8/p: 

b k = l{k + l)/p + 4>\ - [k/p + 4>\. 

This lemma is a direct consequence of the form of the support of the 
bracket sequence a, given in Chapter 2, § 2.2 Note that 8 and p are linearly 
independent over the rationals for almost all 8 in [0, 1). 
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4.6.2 The time slot approach 

Now, we are ready to consider the problem to find the optimal admission 
policy with a given rate. We denote by E CT) «(.) the expectation w.r.t. the 
inter-arrival times and the service times in all nodes in the system. Following 
the technical conditions given in Chapter 1, we have to make sure that the 
expected traveling times to node i under the admission sequence a, satisfy 
the properties given in Lemma 22. Let hi : M — ► M, 1 < i < Q be arbitrary 
non-decreasing convex functions. For simplicity we denote in the following 
WJ,(ai, ■ • • ,a„) instead of hi o WJ,(ai, ■ • • , a„). 

In the next two lemmas, we use increasing and decreasing in the non-strict 
sense. 

Lemma 22. Assume that the inter-arrival times and the service times are 
stationary sequences, independent of each other. The following properties are 
true: 

i- E^WJ^ai, • • • , a n ) is increasing. 

ii- K,sW n (a m— 71+1 1 ,«m) < E C7i5 W^(oi, • • • ,a m ), n<m. 
Hi- E^W^oi,--- ,a n ) =E atS W i m (0,--- ,0,a 1 ,--- ,a n ), n < m. 
iv- E CT ^W^(ai, • • • ,a n ) is multimodular. 

Proof. We prove the four properties in a convenient order. 

i- Using the extended Lindley formula, the expected traveling time 
E^fiWJ^di, • • • ,a n ) is increasing in a*,, k = 1, ■ ■ -n. 

Hi- Let us fix n and m with n < m. Since the inter-arrival times are 
stationary, we can couple the <5's such that VI < j < n, 6^ = 6j™l n _ n = 
6j +m - n . Under this coupling and because the system is initially empty, we 
have: 

Wj,(oi,-- - ,a n ) = W' m (0,--- ,0,oi,-- - ,a n ), 

and, 

E CT Wl(ai,-- - ,a n )=E a W i m (0,--- ,0,ai,-- - ,a n ). 
Therefore, if n < m, 

E^W^ai,-- - ,a n )=K,sW m (0,--- ,0,ai,--- ,a n ). 

ii- This is a direct consequence of i and Hi. 

iv- Theorem 18 shows that W^(oi, • • • , a n ) is multimodular, thus in- 
tegrating over 6, E CT ^W^(ai, • • • , a n ) is multimodular as well. □ 

Now, we choose the cost function g l (a) of the control sequence a = 
(oi , 02 , • • • ) to be the Cezaro sum of the expectation of an increasing convex 
function of traveling time to a given node i, of a potential customer admitted 
to the queue at time T n . We have: 

g'ia) Q ^ lim - £ i^W^ai, a n ). 

N^oo iv 

71=1 
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Note that g l (a) could be called the average traveling time at epochs {T n }. 
In the case the sequence {T n } is a Poisson process, g z (a) is the time-average 
of the traveling times. 

To find which admission policy minimizes the cost </*(.), we recall the 
notation for the bracket sequence with phase 6 and rate p, 



We can restate the general theorems given in Chapter 1 , in the particular 
case studied here. 

Theorem 20. For each sequence a such that 



for any 6 £ [0, 1]. 

Proof. First note that the bracket sequence a p k {6) has an asymptotic rate 
equal to p. Then, the results from 1.3 can be applied, since all necessary 
assumptions on W are satisfied. rj 

4.6.3 The counting approach, the bounded case 

In this section, we use the counting sequence b rather than the admission 
sequence a. Therefore, we rather focus on the function hi o W(i) n (bi, ■ ■ ■ ,b n ), 
which is denoted W^(&i, • • • , b n ) for the sake of notation simplicity. 

Lemma 23. Assume that the service time vectors and the inter-arrival times 
are stationary sequences, independent of each other. Assume that the a\ 
(resp. 6k), for each k £ N and q € Q is bounded from above (resp. from 
below) by R (resp. by D). 

i- Es^W^bi, ■ ■ ■ ,b n ) is decreasing. 

ii- Es iff W* • ■■,&„)> Eff,„>K,(& n - m +i, • ■ ■ , b n ), n > m. 

Hi- Efi, ff W*(&i, • • • ,b n ) = E Sj(J W^(/3, ■ ■ ■ , f), 6 n _ m+ i, ■ ■ ■ ,b n ),n < m, where 
P=\QR/D]. 

iv- E$ )CT >V^(&i, • • • ,b n ) is multimodular. 

Proof. The proof holds by using stationarity of the inter-arrival times and of 
the service time vectors. 

i- Using the extended Lindley's Formula, it is clear that Eg j(7 W^(6i, • • • ,b n ) 
is decreasing. 



a' 



p { e) = [kp + e\-[(k-i)p + 9\. 



(4.31) 




it holds that, 



g\a) > g\aP(e)) 
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ii- Let us fix n and m with n > m. Since the service time vectors are 
stationary, we can couple the service times such that VI < j < to, a^ n _ m = 

a\ m>> = Uj. We also couple the <5's in the following way: 



Vl< j<u(m), 6 {n 2 , , = 0< m) =&. 
Under this coupling and because the system is initially empty, we have: 

W* (&!,■■■ ,b n ) > WUbn-m+U--- ,&„). 

Therefore, the inequality holds for the expected values as well, 

Efi,„W* - ,b n ) >^, ff Wi,(6 n - m+ i,-- - ,b n ). 

Note that properties % and ii do not use the fact that the service time 
vectors and the inter-arrival times are bounded. This assumption is used for 
property Hi. 

Hi- First, we show that under the sequence 62, ■ ■ ■ , b n ), the net is empty 
at the time of the second arrival. 

Let us assume for a moment that the second arrival is infinitely delayed. 
Since the system is originally empty, then after the first arrival, each transi- 
tion in the system will fire exactly once (this is a well known result for Event 
Graphs, see for example [38]). At this time, the system is again empty and 
no transition can fire. An upper bound of this time is given by QR. If the 
next customer arrives in the system more than QR units of time later, then 
it will find the system in the original empty state. 

Now, note that n > f3D by definition of D and taking j3 > QR/D makes 
n larger than QR. 

Finally, using the same coupling as for ii, we get 

E«,„W* - ,b n )=E e ^W t n+1 (P,b 2 ,--- ,b n ). 

An easy induction on n gives the result. 

iv- Theorem 19 shows that Eg W^(6i, • • • , b n ) is multimodular. Therefore, 
Ee } „ W^(6i, • • • , b n ) is multimodular as well. rj 

Similarly to the previous case, we choose the cost function of any control 
sequence b = (61,62,- ••) to be the Cezaro sum of the expected traveling 
times of all customers admitted to the queue, to a given node i: 



1 N 

Y(b)= lim -£lfc,a>V* (&!,...,&„). 



n=l 

We recall that the bracket policy with phase 6 and rate p is 

b r k {6) d = [kr + 0\-[(k-l)r + e\. (4.32) 

We can restate the general theorems given in Chapter 1 in the particular 
case studied here. 
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Theorem 21. Under the foregoing assumptions, for all sequence b such that 

N 



lim -J- V] &n < r, 



tten 7^6) > Y{b r {6)), for any 6 £ [0, 1]. 

Proof. Lemma 21 shows that the counting sequence associated with b r k (6) is 
a bracket sequence with rate r. 

Then, the optimization theory developed in Section 1.3 can be applied 
since all necessary assumptions on E CTj gW are satisfied. q 

Note that Lemma 21 says that the average workload and the average 
waiting time are both optimized by the same admission control sequence 
aT{6). 

In the next section, we will prove that the same optimization result holds 
in the unbounded case. The proof uses strongly the result stated in Theo- 
rem 21. 



4.6.4 The unbounded case 

If the service times and the inter-arrival times are not bounded in the original 
stochastic event graph Q, then, we use a fixed quantity Z and we introduce 
a new system Q z 

clef 

where all services times, a q are replaced by a z ' q = mm(Z, o-9 ). The inter- 

arrival times 8 k are also replaced by 8 Z = max(l/Z, 8k)- In the new system 
Q z , the service times are bounded from above by Z and the inter-arrival 
times are bounded from below by 1/Z. 

In Q z , W z ' z is the compose of the traveling time of the n-th customer to 
the i-th node by a convex increasing function. 

If is a random variable with uniform distribution on [0, 1), 

b T k (&) = f \kr + L(fc - l)r + 0J 

is called the randomized bracket policy with rate r. 

Following the results in Chapter 1 , the time average of E$ )CT W z , is mini- 
mized by the randomized bracket policy, (as well by the one with 6 fixed at 
an arbitrary value). We have for all policy b with rate less than p, 

1 N 

7^(6) = Krn^ - £ Es,a W^(6i, .... b n ) 

71=1 
1 " 

> lim -^E,, CT , W^(6f(0),...,6^(0)) 

N^oo iv 

71=1 
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To prove the optimality of the randomized bracket sequence for the orig- 
inal system Q where the service times and the inter-arrival times are un- 
bounded, we need to let Z go to infinity in the previous inequality. For that, 
we need several technical lemmas. 

Lemma 24. The random sequence b r (0) with & uniform on [0,1) is sta- 
tionary and f(b r k (&) = 1) = r, VJfc. 

Proof. By definition, b r k {0) = [kr + 0\ - [{k - l)r + 6>J. The fact that 
Jg 1 [x + 6\d6 = x for any x implies that 

Vfc, / b r k {6)d6 = r. 
Jo 

□ 

Lemma 25. If {a n } is a stationary sequence of vectors, then {mm(a n , Z)} 
and {max(a n , Z)} are also stationary. 

Proof. The proof follows by definition of stationary sequences rj 

clef 

We define the variable j3 k = J2i=i b%- The inter-admission times satisfy 

J0M-1-1 

r k = 6i - ( 4 - 33 ) 

i=Ph 

One sees that r k is a sum of a random number of random variables. 

Lemma 26. Assume that the process {b k } is stationary, and the process {6i} 
is stationary and independent of {bk}- Then, the process {r k } is stationary. 

Proof. We compute the distribution of the joint process t-i , ■ ■ ■ , r* . It is de- 
termined by the probabilities: 

def 

P(6, b) = P(6 2 = ra 2 , • • • ,bk = n k ,6 bl+ i < 6i, ■ ■ ■ ,8 bl+ni+ ... +nk < 8 ni+ ... +nk ) 
= =%,-■■ ,bk = nk,Si + i < $i, ■ ■ ■ , 6i+ ni +—+n k < 8 ni+ ... +nk ). 

i 

Now, 
P(S,b) 

= ^P(6* = i, -,b k = n k )f{6i+i < 6 1 ,-,6 i+ni+ . +nk < 6 ni+ . +n J 

i 

= ^F(6i =i,-,b k = n k )V{6i < Si,-,S„ 1+ . + „ k < 6 ni+ . +nk ) 

i 

= ^ P(6 =i,h=n 2 , •, b k -i = n*)P(<5i <Si,-, 6 ni+ . +nk < 6 ni+ . +nk ) 

i 

= P(6 1 = n 2 , ■, b k -i = 7ik,Si < 5i, ■,5„ 1+ . +nfc < $ ni +.+ n J, 
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where the first equality follows from the independence of b and 6, the second 
follows from the stationarity of 6 and the third from the stationarity of b. 

This last expression gives the distribution of the joint process n, • • • Tk-i- 
Since all of this holds for all k, the process {r^} is stationary. q 

Lemma 27. Under the randomized bracket policy, if {at} and {6k} are sta- 
tionary sequences, then {<xf } and {r k } are also stationary. 

Proof. The proof comes from a direct combination of the three previous lem- 
mas. □ 

Lemma 28. If {o~ k } and {6k} are stationary sequences and if the system Q z 
is empty originally, then for all n, 

W^(b p (0)) <sW^Ab p (0)), 
where < s denotes the stochastic order. 

Proof. If the service time and the inter-arrival sequences are stationary (as 
it is the case here), this result is well known (see for example [24]). We can 
set W^\b p {e)) = and we have 

W^' q (b p (0)) < s W^(b p (0)). 

Now, the Lindley formula for (max,+) systems can be used in this case. 

Wjf (b p (0)) = A(k - 1) ® D{-T k -i) ® WLi{b p {G)) © B, 

and 

= A'{k) ® D(-r'k) ® W' Z k (b p (e)) ® B, 

where the primes denote another sample path of the service and inter- 
admission times. By induction, we can assume that component-wise, 

w^{b p {e))> s wt,{b p {&)). 

Now, stationarity of the service times in all queues and of inter-arrival times 
comes from Lemma 27. We can couple the service times and the inter-arrival 
times such that for all i, a\ = and r' k = Tk-i- Under this coupling, 

A(k - 1) = A'(k) and D{-r' k ) = D{-T k -i). Hence 

w' z k+1 (b p (0))>w ,z k(b p (e)) 

component- wise, under this coupling. □ 

The optimality of the randomized bracket policy is established by the 
following theorem. 
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Theorem 22. Consider the system Q where the service time vectors and the 
inter-arrival times are stationary sequences, independent of each other. For 
each control sequence b such that 

N 



lim -^7 b n < p 



71=1 

it holds that 



1 N 

f{b) > VLrn^ - X)^,eW^(6T(e), ...,&£(©)). 



71=1 



Proof. Remark that the quantity W% ' z (b) is increasing in Z and 



lim W^(b) = su Pz W^(b) = Wl(b). 

Z — >oo 

The proof follows from the series of inequalities, 

1 N 

>sup z lim -X)Eff,a>V^*(6i,...,6„) 

71=1 
1 ^ 

> su Pz lim - £ E*, ff , W^(6f (0), &£(€>)). 



71=1 



By Lemma 28, we know that E^ ;(Tj 0>V^' l (6f (0), &£(©)) is increasing in 
7i. Therefore, the Cezaro limit equals the supremum on all n. We continue 
the previous inequalities: 



f(b) > sup^sup^eWf'^©), ...,&£(©)) 
= sup n su Pz E ff , ff , e >Vf' < (6?(e) 1 &£(©)) 
1 w 

= lim -^E^, W;(&?(0), ...,&£(©)). 

N—>oo iV — ^ 

71=1 

□ 

Our last theorem shows that the (non-randomized) bracket policy is op- 
timal for any initial phase 6. 

Theorem 23. Under the assumptions of Theorem 22, for each control se- 
quence b such that 



lim — b n < p 

n=l 

it holds that 
for all0<9<\. 
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JV 



Proof. Define 



f m (9, P ) 4i f Es >ff W m {b\ (9) , . . . , b p m (9) ) . 



Note that f m is periodic (in 9) with period 1. Define 

fL(0,p) d = /™K_ m+1 (0),...,ag(0)). 

Then we have 

f' m {0\p) = f m (0,p) where 6' =6- mp, 

Again f' m is periodic w.r.t. with period 1, and is increasing in m because 
of property ii in Lemma 22, so that the following limit exists, 

fL(8,P) d = lim f'JB,p). 

Hence, 

Ee = lim E f' m (0,p) = lim E / m (<9,p). (4.34) 

m — >oo m — >oo 

Now, it follows from our results in Chapter 1 that under properties ii and 
iv of Lemma 22, the following holds: 

1 N 

lim -£E^W;(&?(0),...,6£(0)) =E e f' 00 (e i p). 

n=l 

Combining this with Theorem 22 and Equation 4.34 finishes the proof, q 
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5.1 Introduction 

This chapter discusses the relevance of the assumptions made in the systems 
studied so far for the application of the theory for queuing networks. 

Let us first consider a circuit switched network. We focus on a single 
connection in the network with nodes dedicated to a single class of packets, 
as illustrated in Figure 5.1. 




If we consider networks where nodes are not dedicated to a single connec- 
tion (this is the case in most networks), then we have to take into account 
cross traffic, that is other connections using the same nodes inside the net- 
work. 

In Figure 5.2, cross traffic has been added to the connection under study. 
The goal of the rest of the chapter is to check when the general theory is 
applicable to these cases 
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Fig. 5.2. A network with a connection and two cross traffic 



5.2 Topological assumptions 

As we have seen in Chapter 3, any network (or part of a network) which 
can be modeled by an event graph fits in the framework. This is the case for 
G/G/l queues in tandem as shown in Chapter 3. More generally, this is the 
case for networks of FIFO mono-server queues with finite or infinite buffer 
and general blocking which do not contain any routing. 

In particular this means that all customers must follow the same route and 
that no losses are allowed. We shall investigate in a later chapter (Chapter 
8), admission control and routing into some particular networks with losses. 




queue 3 



Fig. 5.3. A network with routing of the customers 
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Imagine a network as in Figure 5.2. The input sequence is periodic with 
one arrival every time unit. The admission control filters the input with a 
Sturmian sequence with rate 2/3. The first queue has a deterministic service 
rate of one customer per second (therefore the waiting time in this queue is 
always 0). The stream of customers out of queue one is split into two with 2/3 
of the customers sent to queue two (the rest is sent to queue three). It is rather 
easy to check that no matter how the splitting is done, the stream in queue 
2 is not regular. In particular, this stream will contain customers with inter- 
arrival time equal to one. However, if the admission control a is not Sturmian 
but for example a = (1,0,1,1,1,0,1,0,1)°°, then it is possible to split the 
output of queue one with ratio (2/3, 1/3) and shape the stream of customers 
in queue two (resp. queue three) such that a± = (1,0,1,0,1,0,1,0,0) (resp. 
02 = (0,0,0,1,0,0,0,0,1) ) which are Sturmian, and therefore optimal, for 
the waiting times in queues two and three. 

This means that in general, when the network contains several routes for 
the customers, the Sturmian admission may not be optimal. 

Cross traffic as perturbations. When the network is (max, plus) linear, 
the expected (virtual) traveling time of a customer from the entrance in the 
network to its destination, or more generally to any node in the network, is 
multimodular and the general theorems can be used to show that Sturmian 
admission is optimal. 

The example displayed in Figure 5.1 fits in this framework. However the 
example in Figure 5.2 does not since it contains several classes of customers 
with different routes. However this case can be approximated by replacing 
the cross traffic by perturbation of the service in all stations shared between 
the main connection and the cross traffic. 

The idea is to replace the cross traffic by perturbed service times for 
the connection under study: just replace the cross traffic in node i between 
costumers n — 1 and n by a modified service time for costumer n by adding 
the service time of all the cross traffic to the service time of n. This new 
service time is called "visible" service time in the rest of this section. If the 
cross traffic at every node is stationary, then the perturbation at each node 
will also be stationary. However, the service times now become a functional 
of the arrival times for the connection under study. It also induces tricky 
dependences between the services times in the different nodes of the network. 

5.3 Stochastic assumptions 

The stochastic assumptions used in Theorem 21 are the following ones. The 
arrival process is stationary as well as the service times in all queues and the 
arrival process is independent of all the service processes. 

First note that the stationary assumptions are rather minimal. In particu- 
lar, the arrival process does not even need to be ergodic. Without stationarity, 
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the theorem is not true. Just imagine a D/D/l queue with a non-stationary 
fluctuating input sequence with inter-arrival times 

8§n = 1, 
$6n+l = 1, 
8qu+2 = 1, 
^6n+3 = 2, 
8qu+4 = 2, 
^6ti+5 = 2, 

and with deterministic service times equal to 2. If one wants to admit 
optimally a proportion a = 2/3 of the packets, the optimal policy is 
a = (100111) 00 with null waiting time and all Sturmian sequence with density 
2/3 have a positive average waiting time. 

Another interesting property of Theorem 21 is that the different queues in 
the network are not required to have independent service times. For example, 
a "large" packet may have large service times all through the network. This 
could be used as a simplified model for different classes of customers, all 
following the same path and distributed randomly in the global input process. 
It can also be used in order to model the influence of cross traffic for the visible 
service times in several nodes of the network. 

As for the independence assumption between the arrival sequence and 
the service times, this may seem as a rather strong condition imposed on the 
system. It reduces the set of queuing networks where the general theory can 
be applied. For example, the case with cross traffic does induce dependences 
(as shown in the previous section). 

5.3.1 Independence between service times and inter-arrival times. 

If we do not have independence between the service times and the inter- 
arrival time, then Theorem 18 cannot be used because the multimodularity 
in expectation may not hold anymore. Furthermore, one can easily construct 
an example where the optimal admission sequence is not Sturmian. Here is an 
example. Consider a G/G/l queue. We want to admit half of the customers 
in the queue (this means a = 1/2). We consider a Poisson arrival process of 
intensity A. As for the service times, we fix N arbitrarily and we construct the 
service process as follows. The first N customers have service times coupled 
with their inter-arrival times such that Ok = 8k for all 1 < fc < N. As for the 
next N customers, their service times, Ok is exponentially distributed with 
parameter A independent of the inter-arrival times as well as the other service 
times. The next N customers have arrivals coupled with their service times 
again and so forth. Thus, 

u k = 8 k if k < N mod 2JV, 

<7fc ± {8i}i£f$, and Ok i-i-d- exponential, otherwise. 
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In this case, the inter-arrivals are i.i.d. (hence stationary), the service 
times are i.i.d. (hence stationary) but the two processes are not independent 
of each other. 

We consider the following admission policy with admission rate a = 1/2. 



If the queue is initially empty, then for all ieN, the virtual waiting time 
at the ith slot is null: Wi(a>i, ■ ■ ■ a>i) =0. Therefore, 



On the other hand, if one considers the Sturmian admission sequence with 
rate 1/2, to, such that ffiii = 1 and m,2i+i = 0, then E a ,sWi(mi, ■ ■ ■ , rrii) = 
if i < N but for N < i < 2N, E a ,sWi(mi, ■ ■ ■ , rrii) > since in this case 
the queue can be seen as a transient E^/M/l queue and the expectation of 
the waiting time is positive. Therefore, the cost for the Sturmian sequence is 
strictly larger than the cost of the optimal sequence a. 

Note that as N increases, the optimal sequence a get further and further 
away from the Sturmian sequence to. 

This shows that the independence assumption is essential for Sturmian 
sequences to be optimal admission sequences. 

5.3.2 Cross traffic 

As mentioned before, the presence of cross traffic in the network can be 
modeled by perturbation of the service times in each queue. Unfortunately, 
this trick usually introduces dependences between the arrival times and the 
visible service times in the connection under study and the general optimiza- 
tion theorems do not apply in general. However, there are cases when these 
dependences disappear. 

1. This is the case if cross traffic has a lower priority than the connection 
under study (and is preemtable). In this case the visible service time is 
the same as the service time and independence is preserved. 

2. Another case where the independence remains is when all connections 
(cross traffic as well as the main connection) are all deterministic pro- 
cesses, which is rather sensible at the packet level. In this case, the visible 
service can be rather difficult to describe (but remains stationary). The 
arrival process of the main connection being deterministic, it is indepen- 
dent of its visible service times. 



a k = 1 if k < N mod 2N, 
at — otherwise. 




i=i 
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6.1 Introduction 

It is a rather general problem to consider a system with multiple resources and 
tasks. Tasks can be performed by any resource and arrive in the system se- 
quentially. The problem is to construct a routing of the tasks to the resources 
to minimize a given cost function. Such models are common in multiprocessor 
systems and communication networks, where the cost function may be the 
combined load in the resources. 

In this chapter, we show that under rather general assumptions, the opti- 
mal routing policy in terms of expected average workload in each resource is 
given by a balanced sequence, that is, a sequence in which the option to route 
towards a given resource, is taken in an evenly distributed fashion. To show 
the relevance of this type of problem, let us briefly review the recent literature 
on load balancing. Load balancing in a distributed multiprocessor computer 
system has become an important issue to improve their performance. Many 
papers have been devoted to the load balancing problem, and for an overview, 
we refer to [33, 60]. Let us assume there is a centralized controller , and the 
information available to the controller determines the type of control which 
can be used. In dynamic feedback control (or close loop control), the infor- 
mation on the system (e.g. queue sizes) increases as time runs. In static or 
open loop control, the central controller only knows his past actions. Clearly, 
the closed loop models have a better performance than the open loop ones. 

In the dynamic control setting, it is well known that under various as- 
sumptions and cost structures, the "join the shortest queue" policy is optimal 
for homogeneous processors (i.e. all processors are stochastically identical). 
For non-homogeneous processors, an optimal load balancing policy has not 
been found. Only partial optimality results are available (see [62] for results 
on shortest queue policies and [14] for results on the monotonicity structure 
of optimal policies). 

For open loop control, probabilistic routing and pattern allocation have 
been studied. Again, in the case of homogeneous processors, the optimality 
of equal probabilities for probabilistic routing is established in [40, 79]. The 
round robin routing was proved optimal in the case of pattern allocation 
in [82]. For the non-homogeneous case, the problem of finding the optimal 
pattern allocation is generally considered difficult. Approximations of optimal 
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allocations are found in [26, 43]. For markovian models, an algorithm has been 
developed for computing nearly-optimal policies in [65, 64]. 

6.1.1 Organization of the chapter 

Here we will show an application of balanced sequences introduced in 2 for 
routing problems. This uses again results from convex analysis, established 
in Chapter 1. The main results that are used here are of two different kinds. 
First, we use the fact that the workload as well as the waiting time of cus- 
tomers entering a (max,+) linear system are multimodular functions, under 
fairly general assumption (stationarity of the arrival process and of the service 
times, as we showed in Chapter 4). We pose the problem of routing in terms 
of time average of multimodular sequences. Then, we develop the required 
theory for the case where the performance for each subsystem is stochastic 
but its expectation is multimodular. This extends the deterministic analysis 
that we did in Chapter 1, that proves that multimodular functions are min- 
imized by bracket sequences. The superposition of several bracket sequences 
being a balanced sequence, this is the basis of the main result of this chapter. 

It is interesting to exhibit this link between balanced sequences and 
scheduling problems, such as routing among several systems. 

Section 6.2 shows the link between the notion of balanced sequences and 
the optimal scheduling in networks. It is also used to establish the optimality 
of balanced sequences for routing customers is a multiple queue system. Sec- 
tion 6.3 presents special cases for which the optimal rates can be computed. 

6.2 Routing of customers in multiple queues 

Here, we present an application of balanced sequences in arbitrary dimensions 
to scheduling optimization. 

We consider a system where a sequence of tasks have to be executed by 
several processing units. The tasks arrive sequentially and each task can be 
processed by any server. The routing control consists in assigning to each task 
a server on which it will be processed. The routing is optimal if it minimizes 
some cost function that measures the performance of the system. 

These kinds of models have been used to study load balancing within sev- 
eral processors in parallel processing problems as well as for efficient network 
utilization in telecommunication systems, as presented in the introduction. 

6.2.1 Presentation of the model 

In this section we consider a more precise queuing model of the system that 
we described. Customers enter a multiple queue system composed of K sub- 
systems. Each sub-system is made of several queues, initially empty which 
form an event graph (see Chapter 3). 
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The routing of customers to the different sub-systems is controlled by a 
sequence of vectors {a n }, with a n is in {0, 1} K and a\ = 1 means that the 
n-th customer is routed to sub-system i. Note that a is a feasible admission 
sequence as long as for all n, J2i a n = 1- 

The link between a feasible routing policy and an infinite sequence on a 
finite alphabet as used in the first part, comes from choosing the alphabet A 
composed by the letters 

{(i,o,--- ,o),(o,i,o,---),--- ,(<),■■■ ,0,1)}. 

Using this alphabet on K letters, a feasible routing policy can be viewed as 
an infinite sequence on A. 



node 1 



node 2 



node K 



Fig. 6.1. Illustration of the routing of customers in a K node system 



Figure 6.1 shows an illustration of the system we are considering. 
We denote by T n the epoch when the n-th customer enters the system. We 

def 

assume that T\ = 0. The inter-arrival time sequence is {6k} = {Tk+i —Tk}. 
Finally, a 1 ^ will denote the service time of the n-th customer entering the 
j-th queue in node i. 

The sequences {6k} and {<J k ' J } wn l be considered as random processes. 
We also make stochastic assumption on these sequences. The inter-arrival 
time of the customers and the service times form stationary processes, and 
we assume that the inter-arrival times are independent of the service times. 



6.2.2 Optimal routing sequence 

In each sub-system i we pick an arbitrary server s, (which may be the last 
server in the sub-system for example). The immediate performance crite- 
rion for sub-system i will be the traveling time to server s, of a virtual 
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customer that would enter sub-system i at time T n . Under a given rout- 
ing policy, this quantity only depends on the values of the n first routing 
choices. From the routing sequence a, we can isolate the routing decision for 
node i: if a l n = 1 then the customer is admitted in node i and if a l n = 
then the customer is rejected (for node i). We denote the traveling time at 
time T n by W^(a\,--- ,a z n ). We will be more particularly interested in the 
expected value of the traveling time with respect to the service times in all 
the servers contained in node i and with respect to the inter- arrival times: 

; del' 

W n (a{,- • • , <) = E aiT h(W^(a{, ■ ■ ■ , <)), where h : M M is any convex 
increasing function. 

A direct application of the results of Chapter 4 to any single node i shows 
that the function W n (a\ , • • • , o^) is multimodular: 

Theorem 24. Under the foregoing assumptions, the function W n (a\ , o^) 
satisfies the following properties. 

(1) - W n (a\,--- ,a l n ) is multimodular, 

(2) - W^(oi, • • • ,a z n ) is increasing in a\, 1 < k < n, 
(8)-Wjp,--- ,0,ai,--- ,<)=Wi(ai,--- ifm>n. 

Using these properties, we will derive as in Chapter 1 a lower bound 
denoted Bi(a,p) for any routing a% for the following discounted cost. This 
function Bi(a,p) is increasing in a and in p and lower-continuous. 

Let us fix some arbitrary integer, N. We define p z a = (1 — a) YlT=i ak ~ la \- 
Now, using assumptions (1), (2), (3) of Theorem 24, we have, 



oo 

5](i-Q)Q"- i w;(ai,^---<) 

71=1 

JV 

>]T(l-aK-W;(0---,al,---<) 

71=1 

OO 

+ £ (1-q)q"- 1 W i jv (<_jv+i---<) 

7l=JV + l 

N 

= ^(l-a)a- 1 #;(0--- ,a\,---<) 

71=1 

OO 

+ Yl (i-aK^^JvK-JV+ir-- ,ai) 

7t=JV + l 

. JV-1 OO 

> WjI E (! - «) a7i (° <+i)+E (! - «)« n «-JV+2 • • • <+i)) 

71=0 7l=JV 

= W N (a N pl,a N - 1 pl l ,--- ,iQ, 
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where the last inequality follows from Jensen's inequality, since the func- 

~ i 

tion W N is convex, and since the coefficients (1 — a)a n ~ x are nonnegative 
and sum to 1. Define 

Bi(a,p) =supW N (a^a""^,-- - ,p) . (6.1) 

JV 

Note that Bi is defined for a fixed sequence {a\}. Also note that Bi(a,p) is 
lower semi-continuous in a and in p. 
The previous analysis shows that 

oo 

£(1 -a)a n - 1 W , n (ai,t4 ■■■,<) >B t (a,pl). (6.2) 

71=1 

Also, for a given p, we consider the bracket sequence with rate p and 
arbitrary phase (9, 

<W= f \np + 6\ - L(n-l)p + «IJ, 
(see the definition 5). One can show as in Chapter 1 that a v {6) satisfies 

lim -E r nK(«),-,^))=5i(l,p). (6.3) 

m— >oo ffl — 
71=1 

Here, however, we are interested in the performance of all nodes together. 
Therefore, we choose as a cost function, the undiscounted average on n of 
some linear combination of the expected traveling time in all nodes. 

Let h be any increasing linear function, h : R K — ► M. We consider the 
undiscounted average cost of a feasible routing sequence a, 

N 

g(a) d ^\mi N ^^2h(Wl--- ,W*). (6.4) 

Our objective is to minimize g(a). 
Theorem 25. The following lower bound holds for all policies: 
g(a)> inf MB^), ■ ■ ■ , B K {p K )). 

PiH \-pk=1 

Proof. We adapt the method developed in Chapter 1 for our case. We intro- 
duce the following notation. 

B i {Pi ) d = f sup Bi (a , pi ) . 

a<l 

Due to Littlewood's and Jensen's inequalities as well as Equation 6.2, we 
have 
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N 



71=1 

OO 

> n^(i-Q)^Q"- l / l (wL--- ,w*) 

a— >1 f rf 

(oo oo \ 

(1 - q) OL n - x w\, ■ ■ ■ , (1 - a) a n ~ x W K n 
71=1 71=1 / 

> BmAffl^a.ji),--- ,B K (a,p«)) (6.5) 

a — >1 

By definition, we note that J2?=i Pa = 1- Hence, one may choose a sequence 
a„ | 1 such that the following limits exist: 

iim Pa„=Pi, i = l,- - ,K (6.6) 

71 — »00 

and ^2f =1 Pi = 1- From the continuity of Bi(a,pi) in p and cu we get from 
(6.5) 

g(a)>h(B 1 (p 1 ),---,,B K (p K )) (6.7) 



> inf ft(fli(j>i),--- ,fljc(j>jf)). 

PiH hpx = l 



□ 



Note that there exists some p* that achieves the infimum 
inf ,B K (p K )), 

PiH hpx=l 

since h(Bi(pi), ■ ■ ■ ,Bk{pk)) is continuous inp=(pi,-- - ,Pk). 

Consider 6 = (6i, - ■ ■ ,6k) and the routing policy a p (0) given for each i 

by 

ol' P * (ft) = \Mp* + Bi\ - [(k - l)p* + 9 t \ . (6.8) 

There are some p*'s for which the condition of feasibility of the policy 
a p (6) is satisfied, that is, there exists some 6 = (8i, ■ ■ ■ ,0k), such that the 
bracket policy a v (6) is feasible. 

Using the correspondence between a routing policy and a sequence on the 
alphabet A, these p*'s correspond precisely to balanceable rates. 

Theorem 26. Assume thatp* is balanceable. Then a p (6) is optimal for the 
average cost, i.e. it minimizes g(a) over all feasible policies. 

Proof. The proof follows directly from Theorem 25 and Equation (6.3). rj 

Remark 5. The previous theorem says that bracket sequences are optimal 
routing policies. As for balanced sequences (which are ultimately bracket, see 
Theorem 1), they are also optimal if the buffer in each node empties infinitely 
often. This situation occurs when the system is stable as shown in [22]. In 
such cases, a finite prefix of the routing sequence does not alter its cost. 
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6.3 Study of some special cases 

The problem which remains to be addressed is to find in which cases, the rate 
vector p* is balanceable. We will present several simple examples for which 
we can make sure that the optimal rate p* is balanceable. 

6.3.1 The case K = 2 

If K = 2, then, the optimal rate vector is of the form p* = (pl,l — pi). 
Theorem 14 says that p* is always balanceable and therefore, the optimal 
routing sequence is given by an associated balanced sequence. Note that this 
approach does not give any direct way to compute the value of p*, however, 
it gives the structure of the optimal policy. The computation of p* in the 
deterministic case will be done in the next chapter 7 and uses continuous 
fraction decomposition of the service times of both queues. 



6.3.2 The homogeneous case 

Now let K be arbitrary and each node is made of a single server, all servers 
being identical. This model is displayed in Figure 6.2. 




Fig. 6.2. Routing in homogeneous queues. 



Also assume that the function h is symmetrical in all coordinates (for 
example, just the sum of all waiting times) By symmetry and convexity in 
(p u ■ ■ ■ , PK ) of h{V?{ Pl ), • • • , Vk(p K )), we get p* = (1/K, • • • , 1/K), which 
is balanceable. The associated balanced sequence is the round robin routing 
scheme. Applying Theorem 26 yields the following result which is new (to 
the best of the author's knowledge). 

Theorem 27. The round robin routing to K identical -/G/l queues, mini- 
mizes the total average expected workload of all the queues over all admission 
sequences with no information on the state of the system. 
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In [82], the round robin routing is proved to be optimal in separable- 
convex increasing order for K identical ./GI/1 queues. Their method uses an 
intricate coupling argument, whereas our proof is a simple corollary of the 
general theory on multimodular functions. 

To illustrate the advantage of our approach, we further generalize the 
result to a system composed of K identical (max,+) linear systems with a 
single entry. In this case, the symmetry argument used in the case of simple 
queues still holds. Then again, the round robin routing policy minimizes the 
traveling time in each system. This case includes models such as routing 
among several identical systems composed of queues in tandem as shown 
Figure 6.3). 

Fig. 6.3. Routing in queues in tandem 



6.3.3 Two sets of identical servers 

As a consequence of the two previous cases, we can consider a system com- 
posed of Ki identical queues of type 1 and K 2 queues of type 2. Again, assume 
that h is symmetrical in the Ki nodes of type 1 and symmetrical in the K 2 
nodes of type 2. Then, by symmetry arguments, the optimal rate vector is of 
the form 

=1= =1= "1 =1= "I * 

/ P V 1 — V 1 — V \ 

{ k['"' '^'"kT'"' 

for an appropriate p* . This rate vector is balanceable indeed. This implies 
that for the weighted total average expected workload, the optimal routing 
is of balanced type, if nodes of the same type have the same weight. 

Many other examples of this kind can be derived from these examples 
through similar constructions. 
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7.1 Introduction 

The general case of two queues with no state information was studied in 
Chapter 6 where it was shown that the optimal policy in terms of average 
waiting time must be a bracket sequence. However no explicit computation 
of the rate p* of the optimal policy was provided in the heterogeneous case. 

In the deterministic case considered here (both the arrivals and the service 
times are constant deterministic variables), the optimal policy was computed 
in [112] when the system is fully loaded. 

In this chapter, we consider two deterministic FIFO queues with infinite 
buffers and arbitrary constant arrival and service times (hence the load is 
arbitrary). As proved in Chapter 6, the optimal policy is a bracket sequence, 
and we show how to compute explicitly the slope of this optimal sequence, 
which is a rational number as long as the system is not fully loaded, as in 
[50]. Hence, the optimal policy is always periodic even when the parameters 
(service times and arrival times) are irrational numbers. 

In order to obtain this result, we start in Section 7.3 by defining a new 
continued fraction decomposition which helps us for identifying special factors 
of upper bracket sequences. In Section 7.4, we give an explicit formula for the 
average waiting time in one deterministic queue when the arrival process is 
an upper bracket sequence. This function is continuous but not differentiable 
at certain rational points called jumps in the following. It is increasing and 
concave between jumps. Finally, in Section 7.5, we consider the case of two 
queues. Using the properties of the average waiting time in one queue, we 
can show that the optimal routing policy is given by a jump in one of the 
two queues, and hence is periodic, as long as the system is not fully loaded. 
An algorithm is provided that computes the optimal jump in finite time (and 
hence the optimal policy). In the last section, several examples are studied in 
detail in order to illustrate the strange behavior of the optimal policy in such 
a simple system (or so it seems). More on this kind of problems (in particular 
the extension to networks on deterministic queues can be found in [51]. In 
[68], an algorithm is derived to compute a lower and upper bound for the 
average waiting time for the optimal routing for K > 2 deterministic queues. 
A key property is the convexity of the optimal average waiting time as a 
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function of the rate. This is shown in [70] for the general stochastic setting 
od Chapter 4. 

7.2 Bracket words 

We recall the definition of bracket sequences. 

We consider binary sequences over N. In this chapter, to avoid confusion, 
we will adopt the following notations. 

Definition 12 (Bracket sequence). The upper bracket sequence with slope 
a is the infinite sequence m a where the n th letter, with n > 0, is : 

m a (n) = \(n + 1) x a] - \n x a]. (7.1) 

The bracket sequence with slope a is as usual the infinite sequence m a where 
the n th letter, with n> 0, is : 

^(ji) = [(n + 1) x a\ - [n x qJ. (7.2) 

The characteristic sequence of slope a is the infinite sequence c a where the 
n th letter , with n>0, is : 

c a (n) = [(n + 2) x a J - [(n + 1) x a J . 

In the following, by a slight abuse of notation when an infinite word w is 
periodic we also denote by w its shortest period. 

Example 1 (Graphical interpretation). The terminology comes from the fol- 
lowing graphical interpretation. For example let a = 3/7, the bracket word 
is 0010101, the upper bracket word is 1010100 and the characteristic word is 
0101010. 

Consider the straight line with equation y = ax, and consider the points 
with integer coordinates just below the line : P n = (n, [na\), the ones just 
above the line : P' n = (n, \na]) and the points P" = (n, [(n + l)aj). The 
points P n form a representation of the bracket word in the following sense 
: two consecutive points are joined by either an horizontal straight line seg- 
ment, if [w + laj — [na\ = 0, or a diagonal, if L(w + l)aJ — [na\ = 1. Similarly 
the points P' n are a representation of the upper bracket word and the points 
P" are a representation of the characteristic word. In Figure 7.1, the straight 
line b is the line which equation is y = while the upper bracket word is 
represented by a, the bracket word is represented by c and the characteristic 
word is represented by the dashed curve d. 

Remark 6. Since \na] = [na\ + 1 except when na is an integer number, one 
has m a (n) = 2Zia( ri ) when na and (n + l)a are not integer numbers. When 
a is an irrational number, since m a (0) = 1 and m a (0) = then m a = 0c a 
and m a = lc a . When a is a rational number m a is a shift of m a (i.e. there 
exists an integer number k such that main + k) = m a (n) for all n). 



7.3 Expansion in continued fractions 



121 




Lemma 29. Let a be a real number, then the maximum number of consecu- 
tive in m a is [a -1 ] — 1 and the minimum number of consecutive in m a 
is [a^ 1 } — 1. 

Proof. The proof can be found in [84] . rj 



7.3 Expansion in continued fractions 

Let p, < p < 1 be a real number. The computation of its expansion in a 
continued fraction is : 

1 

9 = + ~ • 

m 1 + 

m 2 + • • • ■ 

m n H 

In [84] and [86] one can find an iterative method to compute special factors 
of the characteristic word of slope p and thus build c p , using the coefficients of 
the expansion in continued fraction of p. However, this tight relation between 
the continued fraction of p and the characteristic word c p does not extend 
to the bracket words. For this reason we will introduce a new expansion 
in continued fraction (called upper continued fraction expansion). This new 
continued fraction allows us to find the decomposition in special factors of 
upper bracket words, decomposition which is necessary for the computations 
of the average waiting time. 

The construction of the upper continued fraction expansion of a number 
a with < a < 1 is given by : 

r « = ttt^ • ^ = l« _1 j \ 

1 1 - = i n+1 l an+1 , Wi = L(l - an)- 1 ! , Vn > 1 / ■ 
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When a is a rational number the construction finishes after a finite num- 
ber of steps when l — a n = p. When a is an irrational number the construc- 
tion is infinite and we obtain the infinite upper continued fraction expansion 
of a. 

An infinite upper continued fraction expansion is written under the form 
(h,h, ■ ■ ■ , ln-ijn, • • •)• A finite upper continued fraction is written under the 
form {h,fo, ■ ■ ■ ,ln-ijn)- A partial upper continued fraction of a number a 
is written under the form {h,h, ■ ■ ■ ,l n + ot n ). 

Let us denote by M the set of all upper bracket words and denote by M k 
the set of upper bracket words with slope a such that (fc + < a < fc _1 . 
According to Lemma 29, all words in M k have k or k — 1 consecutive zeros 
between two ones. Let us now introduce the morphism ip k from M k to A*L)A fi , 
which replaces maximal sub-words starting with one and containing only 
zeros according to the following rules 

fc-i 

10^~0 h-> 1 

ifik ■ k 

10^~0 o 



For example m 3 / 7 = 1010100 e Mi, therefore ^2(^3/7) = HO. If a = k~ x 
then the word <Pk(jn a ) is reduced to the letter one. 

In order to generalize <fk let us introduce the morphism 

^ M — > A* U A N 
m a i-)- <Pi a -i\ (m a ) " 

We now show that # has its values in M. 

Lemma 30. Letm a be the upper bracket word of slope a, then $(m a ) is the 
upper bracket word of slope 1 — (a -1 — |a -1 J)- 

Proof. If a -1 is an integer number &(m a ) is reduced to the word "1", then 
we have the result since 1 — (a -1 — |a -1 J) = 1- 

Recall that m a (n) is defined by Formula (7.1). An integer a k is the index 
of the k th occurrence of the letter one in m a if m a (0) . . .m a {ak) contains k 
letters one and m a (0) . . .m a (ak — 1) contains k — 1 letters one. This means 

\a(a k +1)1 = k + 1 , \aa k ] = k . 

These equalities imply a k = [ka^ 1 ]. 

Define now the function ip k from M k to A* U A N which replaces maximal 
sub- words. 



k-l 

10^0 ^ 

k 



10. ..0^ 1. 
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Let I = [a^ 1 ] and let w = ipi(m a ). The sequence (w(k))k>o of letters of w 
can be computed by w(k) = ak+i — (ik — I- Then w(k) = [(k + l)a _1 — l(k + 
1)J — [ka^ 1 — kl\ , hence w is the bracket word of slope a -1 — /. 
Define now the function 7 such that 

1 i-> 
7 : i-> 1. 

The function 7 transforms a bracket word of slope a into an upper bracket 
word of slope 1 — a and conversely. Let w' = 7(w), w' is the upper bracket 
word of slope 1 — (a -1 — /). 

It can be checked that # = 70^), hence #(m a ) is the upper bracket word 
of slope 1 — (aT 1 — |a -1 J)- □ 

The following corollary shows the relation between &(m a ) and the upper 
continued fraction of m a . 

Corollary 6. Let a , < a < 1 be any given real number such that 
a = (Zi, . . . , / n _i, l n + a n ), with n > 1. Let w = &(m a ). Then a(w) = 

+ a n ). 

Proof. By Equations 7.3, we have a = (h + ai) _1 and also 1 — Qi = 
(Z2 , h , ■ ■ ■ , l n -i ,ln + o-n)- By Lemma 30, we obtain 

a(w) = 1 - {a- 1 - La" 1 ]) = 1 - (a" 1 - h) = 1 - Qi . 
Hence a(w) = (/ 2 , h, ■ ■ ■ , L-i, L +a n ). □ 

This will allow us to use induction on the number of terms in the upper 
continued fraction in the following. 

Theorem 28 ((x,y)-factor decomposition). Let a, < a < 1 be any 

given real number with a = {h, fo, ...,/„,.. .). Let us define two sequences 
\xi{a)}i> and {^(a)}i>o, by : 

x (a) = 1, Xi(a) = Xi_i(a)(2/i_i(a)) ii_1 , fori> 1, 
y (a) =0, yi(a) = Xi-i(a)(yi-i(a)) li , fori> 1. 

Then the upper bracket word m a can be factorized only using the two factors 
Xi(a) and yi(a). 

These two sequences are called (x-y)-factor decomposition sequences as- 
sociated with the upper expansion of a. 

Proof. We will study finite sequences {xi(a)}o<i< n , {yi{ot)}o<i<n associated 
with the partial upper continued fraction expansion of a and prove the result 
by induction. 

Step 1. We have a = (h + Qi). By Lemma 29, it is immediate to check 
that m a can be factorized only using x\ and y\ . 
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Step n. We have a = {h,h, - ■ ■ ,l n + ot n ). Let w = #(777 a ), by Corol- 
lary 6 a(w) = {h, h, ■ ■ ■ , l n -i, In + ot n ). Let the two (x-y)-factor decom- 
position sequences associated with the partial upper expansion of a(w) be 
{xi(a(w))}o<i< n -i and {yi{ot(w))}o<i< n -i ■ By induction on the number of 
terms in the partial expansion w can be factorized only using Xi(a(w)) and 
yi(a(w)). Introduce now the sequences x\ and y\ such that 

x'q = 1, y'o =0, ViA x 'i) =Xi-i(a(w)), tpi^yi) =Si-i(aH), Vi > 1. 

Then w = $(m a ) can be factorized only using fi^x'j) and f^iy'i), Vi > 0. 
Therefore to„ can be factorized only using x\ and y^, since #(777 a ) = fi 1 (m a ). 
Note now that the sequences x' i: y[ are the (x-y)-factor decomposition se- 
quences associated with {h,h, - ■ ■ ,l n + (*n)- □ 

Lemma 31. Let a, < a < 1 6e any given rational number with a = 
(Zi, . . . ,/„-i,/„). T/ien x n (a) = 7n a . 

Proof. We also use here an induction on the number of terms in the partial 
upper expansion. 

Step 1. Considering that a = (h) then a is a rational number with a~ x = 
li. Hence m a = 10 ... 0, where the number of consecutive is a -1 — 1 = Zi — 1. 

Step n. We have a = (Zi, fo, . . . , l n -i, l n ), therefore w = #(777 a ) has a 
slope a(w) = {h,---,ln)- Using the induction hypothesis we obtain w = 
x n -i{a{w)). Since x n -i(a(w)) = (p h (x n (a)), then m a = x n (a). □ 

Remark 7. Since $( n \m a ) = <pi n o . . . o <p l2 o yzi(777 a ), then x n {a) and y n (a) 
are the only factors of m a , such that $( n \x n (a)) = 1 and $( n \y n (a)) = 0. 

Remark 8. Since Vn > 0, $( n \m a ) is an upper bracket word and since all 
upper bracket words begin with one, then the sequence Xi(a) is a sequence 
of prefixes of m a verifying 



Example 2. Let a = j|. The upper expansion of a using Equations 7.3 is : 



Thus $(m 8 /ig) = 11011010, #( 2 )(7n 8/ i 9 ) = 10100 and $W(m 8/19 ) = 10. 
Using the (x-y)-factor decomposition we obtain X\ = 10 and y\ = 100, Xi = 
xi and y 2 = zil/i, x 3 = x 2 y 2 and j/ 3 = x 2 (y 2 ) 2 with finally x 4 = x 3 y 3 . 
We can check Theorem 28 and Lemma 31 in this case: 



lim Xi(a) = m. 



(2,1,2,2), and 



TTTg/ig = 1010100101010010100. 




777 8/19 = 10 10 100 10 10 100 10 100 . 
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Theorem 28 and Lemma 31 show the connection between the construc- 
tion of factors of an upper bracket word and the upper continued fraction 
expansion of its slope. This connection could be seen as the analog for up- 
per bracket words of the relation between continued fraction expansion and 
characteristic words presented in [86, 84]. 

In the following, the i th factors of the word m a will be denoted Xi and y, 
when no confusion is possible. 



7.4 Average waiting time in a single queue 

The aim of this section is to compute and to study the average waiting time of 
customers, Ws{w) in a ./D/l/oo/FIFO queue with a constant service time, 
S, and constant inter- arrival times before the control. Using an appropriate 
time scale, the inter- arrival times can be chosen to be equal to one. The input 
sequence w is such that w(n) is if no customer enter the queue at slot n 
and w{n) is 1 if one customer enters the queue at slot n. In the following, 
we will only consider an input sequence which is the upper bracket word of 
slope a, m a . 

Let a, < a < 1 be the ratio of customers sent in the ./D/l/oo/FIFO 
queue. The stability condition of the queue is 

a < ±. (7.4) 

7.4.1 Jumps 

We now show that the computation of the average waiting time Ws(m a ) 
tightly depends on the factorization of m a in (x-y)-factors of S 1-1 . For that 
we will exhibit special rational numbers. 

Lemma 32. Let p = (h, fo, . . . , l n -i, l n , ln+i) be a rational number. 

-i) For all real number a which partial upper expansion is (h,h, - ■■ , l n -i, 
l n + o-n), suc h tho-t ln+i is the smallest integer satisfying 

, ln+1 — 1 

dn < —, , 

ln+1 

then x n+1 (a) = x n (p)(y n (p)) l "+ 1 - 2 and y n +i{a) = x n+1 (p). 

-ii) For all real number j3 which partial upper expansion is (li,h, - ■ ■ , l n -i, 
l n + Pn) with 

</?„<!, 

ln+1 

then x n+1 (P) = x n (p)(y n {p)) k and y n +i{/3) = x n (p){y n {p)) h+1 with k > 

ln+1 — 1- 
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-Hi) For any number 7 which partial upper expansion is (Zi, . . . , l n -i,l n + 
7n), then 1 — 7„ is the proportion of x n (/3) = x n (a) = x n (p) in m 1 (i.e. the 
number of x n (/3) divided by the number of x n (f3) added with the number of 
y n (/3) inm a ). 

Proof. Note that 1 — l/l n +i = (l n +i — l)/Zn+i- Hence the Equations 7.3 lead 
to j3 < p < a. 

-i) For alH, < i < n, we have Xi(p) = Xi(a) and yi(p) = yi{a), since 
the upper expansions are equal until the n th coefficient. Let us compute l n+ \ 
the smallest integer such that 



that implies 



, ln+i — 1 
a n < —j 

ln+1 



ln+1 > Z => ln+1 — 1 — |_i J- 

1 - Ct n 1 - a n 



Hence the upper expansion of a is (h,h, - ■ ■ ,l n , ijn+i — 1) +ot n +i)- Therefore 
x n+1 (a) = x n (p)(y n (p)) l "+ 1 ^ 2 and y n +i{a) = x n (p)(y n (p)) l "+ 1 ^ 1 = x n+1 (p). 
-ii) - < p n implies 



1 



a j — -n+1, 
' Pn 

therefore the number of consecutive y n {P) in x n +i(/3) is larger than l n+ i — 1. 
Since x n (p) = x n (/3) and y n (p) = y n {P) then we proved the result. 

-iii) By Remark 7, x n (^) and y n (n) are the only factors of the word m 7 , 
such that (7)) = 1 and {Vn{l)) = 0. Hence the composition in 

x n{l), 2/n(7) of m 1 is the composition in 1 and of $( n '(m 7 ). By Lemma 30 
and Corollary 6, #( n )(m 7 ) is an upper bracket word of slope 1 — "/ n . Therefore 
using the definition of the slope, 1 — 7™ is the proportion of x n in 7n 7 . rj 

The previous Lemma says that in all bracket words with slope a < p = 
{h, h, ■ ■ ■ , l n -i, InJn+i), the sequence x n (p)y n (p) lr '+ 1 ^ 2 never appears. But 
as soon as the slope is larger than p then the sequence x n {p)yn{p) lr ' +1 ~ 2 
appears. Intuitively, when you increase the slope the number of ones increases 
and the number of zero decreases, hence the minimum number of consecutive 
y n decreases until x n (p)yn(p) l " +1 ~ 2 appears. 

Definition 13 (jumps). Let us denote the upper continued fraction expan- 
sion of S~ x by (Zi, . . . , l n ,- ■ •) and let us define the rational number r^S^ 1 ) 
by r^S^ 1 ) = (Zi, . . . , Zj_i, Zj + 1). The number r^S^ 1 ) is called the i th jump 
ofWs- When S is a rational number, then S~ x = (h,h, ■ ■ ■ ,In) and we 
define the last jump r^^ -1 ) by ^(-S 1-1 ) = (Zi, h, ■ ■ ■ , In)- 

Note that the jumps form a sequence of rational numbers increasing to- 
wards S~ x . When S~ x is rational, the number of jumps is finite and the last 
jump equals S~ x . 
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Let us define rii and qi to be the two relatively prime integers such that 
r^S^ 1 ) = rii/qi. Therefore by the definition of the slope, rii = \m Ti ^ s -ij\i 
and qi = |m ri (s-i)|. For example, if r-2 = {h,h + 1) ; then it can be checked 
that ri2 = /2 + I and qi = /i(/2 + 1) + /2- From now on r^S^ 1 ) will be denoted 
by Ti when no confusion is possible. 

Lemma 33. The number n+i is the rational number with the smallest de- 
nominator in JrjjS 1-1 ]. 

Proof. Let a = (h,h, . . . ,h + «i) be any given rational number in ]n, 5 -1 ]. 
By Equations 7.3 we have [(1 — cni) _1 J > h+i - Hence by Lemma 32, Xi(yi) li+1 
is a factor of m a . That means \m a \ > \xi(yi) li+1 \- Since Xi(yi) li+1 is the 
bracket word m ri+1 then 1 = qi+i- Considering that \m a \ is the 

denominator of a ends the proof. rj 

Remark 9 ( Connection with classical continued fractions) . The rational num- 
ber Cfc , obtained by keeping only k terms in a classical expansion in continued 
fractions is called the k th convergent. The even convergents ci n of a simple 
continued fraction form an increasing sequence. Hence if [mo,mi,ni2,...] 
denote the classical continued fraction of S 1-1 , then the even convergents sat- 
isfy c 2n = [mo,mi,...,m 2n ] < S~\ If c 2n = [mo,ffli,...,m 2 „] is an even 
convergent of S 1-1 , then the rational numbers [mo, mi, . . . , m2 n -i,P2n] with 
P2n € {1,2, .. . ,m,2n} are called intermediate convergents. 

Note now that the sequence of intermediate convergents and even conver- 
gents is the sequence of jumps r^S^ 1 ), since this sequence increases towards 
S~ x and each term of this sequence is the rational with the smallest denom- 
inator in the interval formed by the preceding intermediate convergent and 
S^ 1 and since \c\ + 1] = r±. 

However it seems hard to use the intermediate convergents to compute 
the average waiting time as it is shown further. 

7.4.2 Formula of the average waiting time 

Recall the classical Linley formula for GI/GI/1 queues. Let T n denote the 
epoch of the admission of the n th customer and S n the duration of its service 
time. The workload w* which denotes the amount of service (in time units) 
remaining to be done by the server at the epoch t is given by 

vf = (vjT-' 1 + S 71 - 1 - (t - T"- 1 ))" 1 ", for T"- 1 <t<T n , 

where x_ = \im y -^ x y and (-) + = max(-,0). The waiting time of the n th 
customer is given by 

W n = w T - . 

The decomposition of mV 4 in (x-y) factors of S~ x allows us to compute 
recursively the average waiting time of the input sequence m Ti : Ws{m ri ). 
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We will see that the factors Xi tend to increase the load in the queue whereas 
the factors y, tend to decrease the load. From that, we can compute Ws(m a ) 
for any rational number as well for any irrational number a, n < a < r,+i. 

Let d\ denote the increase of the workload after an input sequence equal 
to Xi. Let d\ denote the maximal possible decrease of the workload after an 
input sequence equal to y,. 

Lemma 34. // the initial workload is equal to zero then using Xi as input 
sequence, the workload during Xi is never null and remains non negative at 
the end of the sequence. If the initial workload is equal to zero then using 
yi as input sequence the workload is never null during yi until its last letter 
(which is always aO) and is null at the end of the sequence. 

Proof. The proof holds by induction on i. 

Step 1. Assume S~ x is not an integer, we have fa < S < fa +1. Therefore 
S — h — 1 < < S — h and the result is proved for step one since the workload 
after x\ is equal to S — h and the workload after y\ is max(5 — h — 1, 0) = 0. 
Moreover Equations 7.3 yield d\ = a\ and d\ = Qi — 1. When S~ x is an 
integer, we have S — fa =0. 

Step 2. By Theorem 28, x 2 is composed by x\ followed by (fa — 1) con- 
secutive yi and yi is composed by X\ followed by fa consecutive y\ . Hence to 
prove the result we have to show that V j, 1 < j < max(/ 2 — 1,1) : 

ai +{fa - j)(ai - 1) > 0, 
ai + fa{ai - 1) + 1 > , 
ai +fa{ax - 1) < 0, 

this last inequality implying the nullity of the workload. By Lemma 32, fa 
is the smallest integer satisfying ai < fa/ {fa + 1), then for all j such that 
1 < j < max(/ 2 — 1,1) we have 

fa > fa- j 

fa + l~ ai fa + 1 - j " 

Since (fa — l)/{fa) > {fa — l)/{fa + 1), the three inequalities are satisfied. 

It can be checked that cti + (fa — l)(ai — 1) = (1 — ai)a 2 and a\ +(fa){ai — 
1) = (1 — ai)(a-2 — 1). Therefore d\ = (1 — ai)a 2 and d\ = (1 — «i)(a 2 — 1). 

Step i+1. Suppose that d\ = (1 — ai) ... (1 — oti-i)(oti — 1) and d\ = 
(1 — ai) ••• (1 — ai-i)oti. We have to show that V j, 1 < j < max(Z; + i — 1,1) : 

(1 - ai) . . . (1 - ai_i) [ai + (h+i - fifa - 1)] > , (7.5) 
(1 - ai) ... (1 - ai-i) [ai + {l i+1 {ai - 1)] < . (7.6) 

These inequalities are satisfied indeed, since k + i is the smallest integer such 
that ai < li + i/(li + i + 1). Moreover, by the induction hypothesis, during the 
sequence yi the workload is positive until the last 0, when the initial workload 
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is null. Consider now the last factor yi in j/i+i, its initial workload is positive 
by Inequality 7.5. If the workload is never null during an input sequence with 
an initial workload equal to zero, then the same result holds with a positive 
initial workload. Hence the workload is never null during the last and also 
during y i+ i , up to its last letter. 
We have 

oti + (h+i - - 1) = (1 - oti)a i+ i, (7.7) 
an + (k+i)(ai - 1) = (1 - ai)(a i+1 - 1), (7.8) 

then we obtain 

d? = (l-ai)...(l-ai_i)(ai-l) (7.9) 
d\ = (l-ai)...(l-ai_i)ai, (7.10) 

This finishes the proof. □ 

Remark 10. When S = {h,h, - ■ ■ ,In) is a rational number, 1 — ajv-i = 1/Zjv 
which is equivalent to qjv-i = (In — 1)//jv- Then ajv-i+(/«-l)(ajv-i-l) = 
0, this yields d x N = . Hence the workload during xn is positive until the last 
epoch of the admission sequence xn(S)~ x = xn(tn), and null at the end. 

We give now a direct consequence of Lemma 34. The workload increase 
d\ and the maximal workload decrease can be recursively computed for 
alH > 2 by : 

d\ = d\_ x + (h - l)d?_i , d\ = d\_ x + (k)dti (7.H) 
with d\= S -h ,dj = S -h-1. (7.12) 

Example 3 (Workload with S = 51/20 with m 5/13 ). Let S = 51/20 = 2.55. 
The partial upper expansion of order three of 20/51 is (2,2,1 + 2/7). The 
(x,y)-factor decomposition gives x\ = 10, y\ = 100, xi = 10100, yi = 
10100100, x 3 = x 2 and y 3 = 1010010100100. Note that r 3 (20/51) = 5/13 
and that m 5 /i 3 = y 3 . 

The Equation 7.11 leads to 

d\ = 0.55 , d\ = -0.45, 
d\ = 0.55 - 0.45 = 0.1 , d\ = 0.55 - 0.90 = -0.35, 
d\ = d\= 0.1 , d\ = 0.1 - 0.35 = -0.25 . 

The figure 7.2 represents the workload during a sequence y 3 . The quantity 
d\ is represented by a, d\ by b, d\ by c, d\ by d and d\ by e. 

We have ^n Ti \\ = \xi-! |i + h\yi-i |i, (|m r Ji = |xjv-i |i + (In - 1)|2/jv-i|i 
when S" 1 = (/i,/ 2 , l 3 , ■■-,/«)) and = \yi-i\i - \yi-2\i- Let S^ 1 = 

(h,. ■ ■ ,h + oti, then the recursive computation of rii = \m ri |i is given by 
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Fig. 7.2. Representation of the workload 



no = 0, ri\ = 1, (7-13) 
Vi > 2 rii = (h + l)m-i - m-2 if a* ^ 0, (7.14) 
rii = k{rii-i) - rii-2 if oti = 0. ( 7 -15) 

This allows us to compute the average waiting time of the input sequences 
m ri recursively. 

The sum of the waiting times of customers admitted during the sequence 
w when the first customer of the sequence waits for a time t is denoted by 
We also denote the sum of the waiting times of m ri over one period 
by K\ and Kf by Ki. Note first, that we can focus on one period since the 
workload after m Ti is null. 

Lemma 35 (Formula of Ws{m ri )). Let S^ 1 = (h, . . . ,k+oti). The average 
waiting time ofm ri is : 

Ws(l r ,) = -, 
with K = 0, K x = and Vi > 1 
if oti ^ 

K i+1 ={k+x + l)Ki - Ki-x + (nik+x - d\ + ni(l i+1 - l)^rf- , 
otherwise 

K i+1 =h+xKi - Ki_x + {ni{k + x - 1) - d\ + ni(l i+1 - 2) ^ +1 2 1 d\. 

(7.16) 

Proof. The minimum number of consecutive zeros in m r , is equal to lx ■ This 
means that the inter-arrival time in the queue is equal to lx + 1 and since 
S < lx + 1 none of the customer has to wait, therefore Kx =0. 
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Recall that we have by Definition 13 (yiY 2 - The waiting time of 

a customer is the workload at the epoch of its arrival. By Lemma 34 the first 
customer admitted does not wait, on the other hand the next fa customers 
will have to wait. Moreover the workload during m T2 is positive until the last 
hence the waiting time of the second customer is d\ , the one of the third 
customer is d\ + d\ , the one of the j th is d\ + (j — 2)d\ and the one of the 
last customer, which is the (fa + l)*' 1 , is d\ + (fa — l)d\. Summing we obtain 
K 2 = fad\ + (fa - l)(J 2 /2)d?. 

Let w be an admission sequence during which the workload is never null if 
the initial workload is equal to zero. When the first customer of w waits for a 
time t, the waiting time of a customer admitted during w is its waiting time 
when the first customer does not wait increased by t. Therefore A'*(u>) = 
(Hi x t) +K°(w). 

We have m ri = 2^-1(^-1)^. Let us compute the waiting times of the first 
customers of the fa consecutive j/i-i. The waiting time of the first customer 
of the first yi-i is d]^. The waiting time of the first customer of the second 
2/i 1 is d]_! + d\_ x . The waiting time of the first customer of the j th yi-\ is 
d\_ x + (j — • The waiting time of the first customer of the last yi-i (the 

lf)\sd\_ 1 +(l i -l)d 2 i _ 1 . 
We decompose Ki. 

Ki -K (Xi-xj + K^ +K i _ 1 +...+K i _ 1 

= Jsr°(xi_i) + rn-xd}^ + + m-^d}^ + d 2 ^) + K^ x + ... 

+ rii-i(rf-_! + (k - + i^i 1 , 

= K°(xi-i) +kKi-i + ni-i(kd]_ 1 + (h - l) l jd 2 _i) , 

since yi-x = ^-i2/i-2, 

Kt = Kt-! - Kf- 1 + UK,-! + m^kdt, + (k - , 

= - (ni_ 2 dj_! + Ki- 2 ) + UKi-i + ni_i(/idj_! + (h - l)^d?_i) • 

When S~ x = {fa, fa, . . . , In), the N th jump is a special case since 
m rw (5-i) = m s -i is xn and the number of consecutive yjv-i following xn-i 
is /jy - 1. □ 

We define the average waiting time of a finite word w by 

... , . K°(w) 

yvs(w) = 11 

We extend this definition by defining the average waiting time of an infinite 
word w. Let w n be the prefix of length n of w, then 
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WsH = lim , 

71^00 \W„\i 

if the limit exists. 

Theorem 29 (Formula of Ws{m a )). Let Ti be the i th jump of Ws{m a ), 
and n+i the i + 1 th jump ofWs{m a ). Let a = (h, - ■ ■ ,h be any number 
such that a £ (r^ri+i). Then the average waiting time ofm a is 



,., ,_ ^ (1 - ai)K i+1 + [on - h+i (1 - Qi)] JiTi 

KVs^maJ = — r — = : r= 

(1 - ai)n i+1 + [cti - (1 - qJJ rii 



(7.17) 



When a is smaller than r\, then Ws(m a ) = 0. 

Proof. The proof is given in Appendix 7.7. rj 
7.4.3 Properties 

Here, we will give some properties of the function Ws(m a ). Most of them 
are required to compute the optimal ratio in the case of two queues. 

Lemma 36 (Continuity). The function a i-> Ws{m a ) is continuous. 

Proof. Let a = (Zi ,...,/„ + a n ) and /?= (Zi ,...,/„ + /3„) be two points in 
the interval [r-ji,/ 1 ,!-)-!]. If |q — /3| goes to zero, then \a n — fi n \ also goes to zero. 
By Equation 7.17, \Ws(m a ) — Ws{mp)\ goes to zero. rj 

Using similar arguments, one can show that the function a — ► Ws(m a ) is 
infinitely differentiate on each interval ]r„,r„+i[, but it is not differentiable 
at the jumps. 

Lemma 37. The function ^ is strictly increasing. 

Proof. The proof is given in Appendix 7.8. □ 

Remark 11. As proved in [112] if S is an irrational number then the average 
waiting time = 1/2. Hence the sequence ^ goes to 1/2 when i 

goes to infinity. 

Theorem 30. If a s]^, Ti+i [, then 

— i) the function Ws{m a ) is strictly increasing in a, 

— ii) the function a i-> Ws{m a ) is strictly concave, 

— Hi) the function a i-> aWs{ot) is linear in a. 

Proof. The proof is given in Appendix 7.9. rj 
To illustrate the results obtained in this section let us take an example. 
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Example 4 (Curve ofW^/^a)). The upper expansion of 4/5 is (1,1,1,2) and 
m 4 / 5 is equal to 11110. The factor decomposition is : X\ = 1 and y\ = 10, then 
X2 = xi = 1 and yi = xiyi = 110, then x 3 = xi = 1 and 2/3 = X2y2 = 1110, 
then £4 = X32/3 = 11110 = m 4 / 5 . 

We have n(4/5) = (2) = 1/2, r 2 (4/5) = (1, 2) = 2/3, r 3 (4/5) = (1,1,2) = 
3/4 and r4(4/5) = (1,1,1,2) = 4/5, therefore ri\ = 1, 712 = 2, 71 3 = 3 and 
714 = 4. 

Let us now compute i^, we obtain Ki = 0, K2 = 1/4, i^3 = 3/4 and 
K 4 =3/2. Hence we have : 

K 1=Q K2 = l «3 = 1 ^i = 3 

rii TJ2 8 ' 713 4 ' 714 8 

Recall that a = (Zi +Qi) = (/i,/2 +02) = (li,h,h +013). Formula 7.17 leads 
to 

Ws{m a ) = , fora < ^. 
W s {m a ) = , for ^ < a < - O ^ < ai < 1. 

W *( W «> = I^TI) , for ^ < a < Jo i < a 2 < I- 

Ws(Wa) = 4(2^7+1) ' f ° r i ^« ^ ^ ^ «3 < I- 
The function Ws(TO a ) for all < a < 4/5 , is shown in Figure 7.3. 




Fig. 7.3. Curve of Ws/4(m a ) when a varies from 0.45 to 0.8. 
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7.5 Average waiting time for two queues 
7.5.1 Presentation of the Model 

From now on as presented in Figure 7.4, we will study a system made of two 
./D/l/oo/FIFO queues, with constant service times Si in queue 1 and 52 in 
queue 2. It is assumed for convenience in the following that 52 is the smallest 
service time. The time unit is chosen such that inter-arrival time slots are 
constant and equal to one. When they arrive, the customers are routed to a 
queue where they wait for treatment in a FIFO order. The ratio of customers 
sent in queue 1 is a and therefore, the ratio of customers sent in queue 2 is 
1 — a (i.e. no customer is lost). Our aim is to find a policy which minimizes 
the average waiting time of all the customers. The problem consists in finding 
an optimal allocation pattern and to find the optimal ratio (q,1 — a). The 
optimal allocation pattern was found in Chapter 6 but no way to compute 
the optimal ratio is provided there. 



Arrivals 




Fig. 7.4. Admission Control in two ./D/l queues 



The input sequence w is given under the form of a binary sequence w such 
that when w(n) = 1 the n th customer is sent in the first queue and when 
w(n) = the n th customer is sent in the second queue. The average waiting 
time of all the customers for an input sequence w is denoted by Ws 1 ,s 2 ( w )- 

7.5.2 Optimal Policies 

To find an optimal allocation pattern we will use results shown in the previous 
chapters. 

Theorem 31. The average waiting time Ws 1 ,s 2 i w ) * s minimized for a bracket 
input sequence m aopt , with some slope a opt . 
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Proof. This is a mere application of the multi-criteria optimization found in 
Chapter 1 in the deterministic case. rj 

Note that the approach proposed in Chapter 6 does not provide any means 
to compute a opt , which is what we are going to do here for deterministic 
queues. Moreover, the routing policy described here is optimal among all 
routing policies, including J.S.Q. (Join the Shortest Queue) and J.S.L (Join 
the Shortest Load), since the distinction between closed loop and open loop 
policies is irrelevant for deterministic systems. Note however that it may not 
be the unique optimal policy. 

Remark 12. It can be shown that the average waiting time in a deterministic 
queue under an arrival process of the form m a is the same as the average 
waiting time when the arrival process is of the form m a . Therefore, Theorem 
31 applies with bracket as well as upper bracket sequences. This remark is 
essential here since when the input sequence in the first queue is an upper 
bracket word of slope a (m a ), then the input sequence in the second queue 
is a bracket word of slope 1 — a (m 1 _ ct ). 

Using the considerations of Remark 12, and conditioning on the queue 
chosen for each customer the relation between W5 li 5 2 (m a ), Ws^m^) and 
Ws 2 (rai-a) is : 

W 5l ,s 2 (ra a ) = a ■ W Sl {m a ) + (1 - a) • W S2 (™i-<*) • (7.18) 



7.5.3 Optimal Ratio 

We will now compute the optimal ratio a opt associated with a bracket input 
sequence over all possible stable ratios. 

Stability Condition Consider the system of the two ./D/l queues above, 
the stability condition of such systems is p < 1 , that is 

^ + ^>1- (7-19) 

But the stability of the two queues individually is also necessary. Therefore 
by Equation 7.4 a^S 1 ^ 1 an d 0<1 — a<S' 2 ~ 1 . Hence for any given number a 
in the interval I s = |l — -^h, ^-j n [0, 1] the input sequence m a is stable. In 
the following the interval I s is called the interval of stability. 

Special Cases This part considers special degenerated cases where the the- 
ory developed in the previous section is not necessary. 

Lemma 38. In the case 2 < S2 < Si, no real optimal policy exists. 
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Proof. The system is never stable since S 1 1 < S 2 1 < 1/2, implying 



1 1 , 

s; + s- 2 ]<1 



□ 



Lemma 39. In the case S2 < 1, an optimal policy consists in sending all 
the customers in the second queue. In this case Ws 1; s 2 (mo) = 0. 

Proof. When S2 < 1, the service time is smaller than the inter-arrival time. 
Therefore each time a customer arrives the second queue is empty and we 
have Ws 2 (l) = 0. Since W 5l (0) = then by Equation 7.18, W Sl ,s 2 (rn ) = 0. 

□ 

Lemma 40. If 1 < S2 < S± < 2 then the round robin policy is an optimal 
policy. 

Proof. Lemma (35) implies Ws^rni/2) = and Ws 2 (toi/2) = 0. Therefore a 
possible optimal ratio is a = 1/2 and TO1/2 = 10 is an optimal policy. q 

Case 1 < S 2 < 2 < Si This can be considered as the general case. 

Characterization of optimal ratios when p < 1 We are interested to find the 
optimal ratio a opt defined by : 

a op t = argmin yy Sl ,s 2 (m a ). 

Let r\ and r 2 - denote the jumps of Ws x and Ws 2 respectively. 

Theorem 32. For any real service time Si and S2 the optimal ratio a opt is 
a jump of or Ws 2 ■ Hence, it is a rational number and an optimal routing 
policy is periodic. 

Proof. By Theorem 30, the function aWs^TOa) is linear for a in 
and the function (1 — a)Ws 2 {mi- a ) is also linear for a in ]1 — rj, 1 — 
Therefore by Equation 7.18 the function Ws 1 ,s 2 is linear for a in }r* k , r* k+1 [. 
Where {r* k } = ({rj} U {r-}) n I s is the set of jumps of Ws u s 2 - 

Hence Ws!,s 2 reaches its minima in the set of all the jumps r k or in S^ 1 
or in S2 1 when they are irrational. Theorem 42 says that the growing rates 
of the function i — > ^ converges to infinity. This allows us to exclude the 
points S^ 1 and S^ 1 - 

Therefore Ws 1 ,s 2 is minimized by a jump and since all the jumps are 
rational the optimal ratio a opt is rational. This means that the routing policy 
m a opt is periodic. □ 

Lemma 41. Let p be the rational number with the smallest denominator in 
the interval I s . Then p is the smallest jump of in I s and the largest jump 
ofWs 2 inl s . 
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Since p is the unique common jump of Wsj and Ws 2 , we call p the double 
jump. 

Proof. Let be the largest jump of Ws^m,,) such that r| < 1 — 1/52- 
This implies 1 — S 1 ^ 1 < r\ +1 < 1/Si. By Lemma 33 r\ +1 is the rational 
number with the smallest denominator in ]rj, I /Si], then r\ +1 is the rational 
number with the smallest denominator in I s . Let r 2 - be the largest jump of 
Ws 2 (1 — a) such that 1 — r| > 1/Si . This implies 1 — r| +1 is the largest jump 
of Ws 2 (l — a) in J s and the rational number with the smallest denominator 
in I s . Then r\ = l- r] +1 . □ 

The double jump is not always the optimal ratio, as we will see in the example 
below. 

Example 5 (Double jump is not optimal). Let Si = 21/5 and 52 = 6/5. The 
upper expansion of 5/21 is (4,1,1,1,2) and the one of 5/6 is (1,1,1,1,2). 
We have r\ = 1/5, r\ = 2/9, r\ = 3/13, r\ = 4/17, r\ = 1/Si = 5/21 and 
l-r-2 = 1/2, \-r\ = 1/3, l-rl = 1/4, \-r\ = 1/5, \-r\ = l-l/5 2 = 1/6. 
The double jump is 1/5. 

The following numerical values have been obtained using exact computa- 
tions provided by the program presented later (Section 7.5.4 and 7.6). 

W Sl {mi/ 5 ) = 0, Ws 2 (to 4 / 5 ) = to, W 5l ,5 2 (mi/ 5 ) = To x f = 

W 5l (m 2 /9) = to, Ws 2 {m 7/9 ) = ^, W Sa ,s 2 (m 2 / 9 ) = rV x | + ^ x | = |- 

Since | < ^, the optimal ratio is not the double jump. It can also be 
shown that the optimal ratio in this case is a opt = |. 

0.45 | , , , , , , , , 1 



0.35 - 

0.3 - 

0.25 - 

0.2 - 

0.15 - 

0.1 - 

0.05 - 

- 

0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3 0.32 0.34 

Fig. 7.5. Curves of Wsj, Ws 2 and Ws 1 ,s 2 when a varies from 1/6 to 1/3. 




e 



138 7 Optimal routing in two deterministic queues 



Figure 7.5 displays the functions Wsi(m ) (curve b), Ws a (mi_ a ) (curve 
c) and Ws 1 ,s 2 (™a) (curve a), as well as some jumps of those three curves 
(the double jump 1/5 is marked by the dotted line d, and the optimal jump 
2/9 by the dotted line e). The vertical lines f show the interval of stability of 
the system, Ws 1 ,S2(™a) is infinite outside this interval. 

Characterization of optimal ratios when p = 1 In this part, we will assume 
that the system is fully loaded. When p = 1 then the stability interval is 
reduced to a single point. There is just one ratio a satisfying the stability 
condition, a = S^ 1 . Therefore the upper bracket word with slope a is opti- 
mal. This is the only case where the optimal policy associated with bracket 
input sequence may be aperiodic. 

7.5.4 Algorithm and computational issues 

We present in Figure 7.6 an algorithm to compute the optimal ratio a op t, 
when p< 1. 

Find double jump p 

current-jump :— p 

Compute the next-jump-right of p 

while W Sl ,s 2 (m current .j ump ) >Ws 1 ,s 2 (m nex t-jump-right) do 

current-jump: = next-jump- right 

Compute the next-jump-right of current-jump 
endwhile 

Compute the next-jump- left of p 

while Ws a ,s 2 (m current .j ump ) >W Sl ,s 2 (m n ext-jump-left) do 

current-jump: = next-jump-left 

compute the next-jump- 1 eft of current-jump 
endwhile 

return current-jump 

Fig. 7.6. Algorithm computing a op t 



Correctness of the algorithm We will now show that the algorithm is 
correct and converges to the result in a finite number of steps. For that we 
have to prove two preliminary lemmas. 

Lemma 42. The growth rate of the function i— ► ^ converges to infinity 
when i goes to infinity. 

Proof. Let S~ x = {h, ■ ■ ■ ,k + cti), the recursive computations of qi = \m ri \ 
are given as for m, by 
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qo = l, 

qi=h + l, ifai^O, qi=h, if Qi = 0, 
Vi > 2 g. = (/. + - g.-2, if a* ^ 
% = (ii)g.-i - <7i-2 otherwise . 

We now show by induction that > 1 

rii+ift - niq i+1 = 1. (7.21) 

At step one : 7i2<7i - n\qi = (fe + + 1) - (h(h + 1) + fe) = 1- 

At step i + 1 : TJ i+ i% - n^+i = + 1)^71* - ft^-i - + l)<£n; - 

Hence the growing rate 

( Ki+1 _ f ni+1 _ 1 - ( n i K *+l ~ ^i+j^Kg^+lgi) 

> (n,-ifi + i - n i+1 Ki) 

> (m-iKi -mKi-i) +m{m -rii-i)d\ 

> £i<k<i (n k (n k - n k -i)d\) . 

Let us prove now that S±< k (n k (n k — n k -i)d\) = +oo. We use Equations 
7.10 that yields 

n k {n k - n k -\)d\ = n k (n k - n k -i) [(1 - Qi)(l - q 2 ) • • • (1 - a k -i)] a k . 

By Equation 7.14 we obtain 

- m-i) = [(h + l)Tij_i + (m-i - TJj_ 2 )] [hrii-i + (nj_i - ri;- 2 )] 
>[h + l + h + h{h + 1)] rii-i{ni-i - ri;_ 2 ) • 

Hence 

< ^ ((J* + !) 2 + **) + !) 2 + • • • + !) 2 + feja* 
- n k -i)dk > — — ; — 

«2«3 • • • «fc-l 

> ((/ fc + 3)) + 3)) . . . ((/ 2 + 3))a* > 4*-^* • 

Two cases may occur, either the sequence {a k } k >o does not converge to zero 
or it converges to zero. In the first case the sequence n k (n k — n k -i)d\ does 
not converges to zero and the series E k >$n k {n k — n k -i)d\ goes to infinity. In 
the other case there exists a number N such that Vfc > N ,a k < 1/2. Since 
a k < 1/2 implies l k +i = 1 and (1 — a k )a k +i = a k then 

n k+ i(n k+ i - n k )d\ +1 _ n k+1 (n k+1 - n k ) ^ 
n k (n k - n k -\)d\ n k {n k -nk-\) ' 

which implies that the sequence is strictly increasing and that the series 
diverges to infinity. rj 
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Lemma 43. The function i — > ^ is convex. 
Proof. We have by Equations 7.16 and 7.21 

( Ki+1 _ ( ni+1 _ Hi\ 1 _ f^i _ Ki - 1 \ (v±- Ui - 1 

V Qi+i Qi J \qi+i (li) \Qi Qi-i J \Qi 

(K i+1 Ki\ (Ki Ki-A 

(Qi+iQi) (QiQi-i) = 

V Qi+i H J \Qi Qi-i J 

qiK i+1 - q i+1 Ki - q^Ki + qiK^x . 
Using Equation 7.16 leads to 

qiK i+1 -q i+1 K i = q i ((l i+1 + l)Ki - K^) - q i+1 Ki 

+ q { (ni(l i+1 - l){d] + ^d?) + (m - m-M) 
= Qi{ih+i + l)Ki - Ki-x^J - (k+iqi - qi-i)Ki 

+ qi (n % {k+i - i){d\ + l -^-<%) + (m - m-M) 
= q i - 1 K i - qiKi- i+qi(ni(l i+1 -l) {d\ + ^±1 d • ) + (m - m-i ) d • ). 

Hence 

q i K i+1 -q i+1 K i -{q i ^ 1 K i - qiK^x) 

= q% (ni(h +1 - i)(d] + ^d?) + (m - m-M). 
As shown in proof of Lemma 37, 

m(l i+1 - \){d\ + l -^-d}) + (m - m^d] > . (7.22) 

This proves the convexity of the function. rj 

Lemma 44. The algorithm is correct and converges in a finite number of 
steps 

Proof. Correctness. Since by Lemma 43 the function i — ► ^ is convex, then 
by Equation 7.18 the function k — ► Ws,,Sj(m r j) is also convex. Considering 
that a opt £ {rj*} shows the correctness of the algorithm. 

Finiteness. Let k,i,j be the integers such that p = rk = r\ = 1 — r?. 
Let us note that 1 — r j < Si < 1 — r^_ x . Since Vn > 1, r* k+n = rj +n , and 
since by Lemma 42 the growing rate of the function i — ► Ws 1 {m r i) goes 
to infinity then by Equation 7.18, > 1 the growing rate of the function 
k+n — ► Ws 1 ,s 2 (™ri +n ) converges to infinity. Therefore the integer fco = k+no 
such that Vn > no 
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Ws„s a (?n r . + J - W Sl , S2 (m r . +n i ) > 0, 

is finite. Using similar arguments for the jumps smaller than p shows that 
the algorithm converges in a finite number of steps. rj 

Concerning this algorithm, which was run hundreds of time with irrational 
or rational parameters, the maximal number of steps never went over twenty. 
Nevertheless one can get a large number of steps with well chosen parameters 
for which the optimal jump is arbitrarily far (in number of jumps) from the 
double jump. 



7.6 Numerical experiments 

The algorithm presented above has been implemented in Maple in order to 
keep exact values for all the rational numbers involved in the computations. 
This section is dedicated to the presentation of several runs of the program 
in order to shown how the optimal policy (or equivalently the ratio of the 
optimal policy) behaves with respect to the parameters of the system, namely 
Si the service time in queue 1, 52 the service time in queue 2 as well as the 
inter-arrival time (fixed to one previously, but which can be modified by 
scaling the time units). 

In the first series of computations, we fix Si = 22/5 and 52 = 6/5. 
Therefore, the fastest server, 52 is c times faster than Si, with c = 11/3. One 
could expect that the optimal routing policy sends c times more customers 
in the second queue than in the first queue. This policy has a ratio a = 
jqrf, namely a = 3/14. However, as the experiments in Figure 7.7 show, 
the optimal ratio is a opt = 1/5 when the inter-arrival time is one (when 
p = 66/70 w 0.942). 

In Figure 7.7, with Si = 22/5 and 52 = 6/5, we let the inter- arrival 
time vary so that the total load p goes from to 1. All the results presented 
in the figure are exact computations and do not suffer from any numerical 
errors. When p is smaller than 11/14 (dotted line on the left), many ratios 
are optimal. For instance, for all a < 1/4, W5 li 5 2 (m ,) = 0. This part is 
not shown in the figure. The main interest lies within the bounds 11/14 < 
p < 1 where the optimal ratio is unique. The optimal ratio a opt takes several 
rational values, ranging from 1/4 to 1/5 and ending with the intuitive ratio 
of the service times, 3/14. When p = 1, then a = 3/14 is the only point 
in the stability region. However, for lower values of the load, it is somewhat 
surprising that the optimal ratio changes with the load and deviates from 
the ratio of the service times, 3/14. The changes of the optimal ratio occur 
according to several reasons. For example, a opt moves away from 1/4 only 
when 1/4 gets out of the stability region (this occurs at p = 6/7). The same 
phenomenon occurs when a opt jumps away from 1/5, which occurs when 
p = 55/56. On the other hand, the change from 2/9 to 1/5 occurs while 2/9 
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Fig. 7.7. Optimal ratio when the total load varies from 11/14 to 1 



is still a jump in the stability region. In general, the sudden changes of the 
optimal ratio when the load increases remain mysterious and deserve further 
studies. 

The second set of computations is presented in Figure 7.8. The inter- 
arrival is fixed to one and we let the inverses of the service times 1/ S\ and 
1/52 vary. We restrict our investigations to the domain of stability that is 
4h + ^ > 1. The figure displays the points where the value of a opt changes. 
Therefore, each cell corresponds to couples (1/ Si,l/ S2) with the same opti- 
mal ratio. The larger cell (a opt = 1/2) corresponds to the region where the 
round robin policy is optimal. Using time homogeneity, the optimal ratios 
along the line = (bold line in Figure 7.8) are those given in Figure 
7.7. We can note that along the line ^- + = 1, the changes of a opt exhibit 
a fractal behavior which also deserves further investigation. 

Figure 7.9 is the same as Figure 7.8 where the cost to be optimized is not 
the average waiting time of the customers but the average sojourn time of 
the customers, Ss 1 ,s 2 {™a) = Wsi,s 2 (™a) + aSi + (1 — a)S2- The optimal 
fraction of customers sent to queue 1 changes. Roughly speaking, the fastest 
queue receives more customers when the sojourn time is used as the cost to 
be minimized instead of the waiting time. 
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We will first assume that the number a is a rational number. Thus we work 
with only finite words. 
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1/S, 1 

Fig. 7.8. Optimal ratio when I/S2 and 1/Si vary 

By Theorem 28, m a can be decomposed only using Xi + i(a) and yi + i(a). 
Since by Equations 7.3 

k+i , ^ -1 

1 — +7 ^ 1 ' 

+ 1 

then we obtain the composition of Xi+i(a) and yi + i(a) with Lemma 32: 

x i+1 (a) = Xi(r i+1 )(yi(r i+1 )) k and yi+i{a) = Xi(r i+1 )(yi(r i+1 )) k+1 
with > (h+i +1) — 1. Let us rewrite these two equalities into : 

x i+1 (a) = x i (r i+1 )(y i (r i+1 )) h+1 (y i (r i+1 )) k ' , 
y i+ i(a) = x i {r i+ i){y i {r i+ i)) h+1 {y i {r i+ i)) k ' +1 , 

with k' > 0. We have Xi + i(a) = m ri+1 (m ri ) k ' and yi + i(a) = m ri+1 (m ri ) k ' +1 . 

Since by Lemma 34 and Remark 10 the workload after m ri is always null, 
then the workload after an admission sequence Xi + i(a) or yi + i(a) is also null. 
Therefore the workload after one period of m a is null and we simply have to 
focus on one period of m a . Computing K°(xi + i(a)) and K°(yi + i(a)) gives 




K°(x i+1 (a)) = K i+1 + k'Ki and K°(y i+1 (a)) = K i+1 + (k' + l)Ki . 

We write a, = n/q, cti being rational. The number of m ri+1 in m a is 
the number of Xi(S~ x ) = 2^(7^+1) in m a . By Lemma 32 the total number of 
a;j(S _1 ) in m a is equal to the total number of ones in $W(m a ). Since 1 — n/q 
is the slope of and since m a and #^'(m a ) are finite words. Then 

the total number of Xi(S~ x ) in m a is (1 — ^)q. Using a similar argument the 
total number of m ri in m a is {^)q- The number of m ri which do not belong 
to m ri+1 is equal to q(j) — h+i{l — j)q- Therefore 

K(m a ) = q(l - ^)K i+1 + q[- q - h +1 (l - ^)]Ki. 

Compute now n a the total number of ones in m a . This number is the 
number of m Ti+1 multiplied by the number of ones in m ri+1 added with the 
number of m Ti which do not belong to m ri+1 multiplied by the number of 
ones in m Ti : 

Tt Tt Tl 

n ai = 9(1 )n i+ i + q[ h+i{l ■ 



We obtain 
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"""^ (l_n )ni+1 + [ n_( /i+1 (l_n)) ]ni " 

Suppose now that a is an irrational number, the bracket word is now an 
infinite word. However we work with finite prefixes which properties are kept 
when we compute the limit. 

Let n be an integer. Let (m a ) n be the prefix of length n of m a . Let 
{ma)s„ be the greatest prefix of m a uniquely composed by factors Xi+i(a) 
and yi + i(a) which length is smaller than n, and let (m a ) gn be the smallest 
prefix uniquely composed by factors Xi + i(a) and yi + i(a) which length is 
larger than n. We obtain 

K°((m a ) e „) < K°((m a ) n ) < K°((m a ) g „) 
\{m a ) g Ji ~ |(m a )„|i ~ \(m a ) Bn \i 

The prefix (m a ) gn is composed by (m a ) Bn followed by either Xi + i(a) 
or y i+ i(a). As shown above, the workload after any sequence Xi + i(a) and 
yi + i(a) is null. Hence the workload after (m a ) gn and after (m a ) Sn is also null. 
Since the factor yi + i(a) is longer than the factor Xi + i(a), then |x; + i(a)|i < 
and K°(x i+1 (a)) < K°(y i+1 (a)). That yields K°((m a ) gn ) < 
K°((m a ) Sn ) + K°(y i+1 (a)) and also |(m a ) 9 „|i < |(m a ) a „)|i + |^ +i (a)|i. 
Thus 

^((mJQKm^j! < i^°((7n a ) 71 ) < if°((m a )Q K°(y i+1 (a)) 

|(m a ) s „|i(|(m a ) s „|i + |^+i(a)|i) ~ |(m Q )„|i ~ |(m a ) s Ji |(m Q ) a „|i 

(7.23) 

Let us focus on K°((m a ) Sri ). For all n > rii+i, this sum is strictly positive 
and only depends on the number of yi + i(n+i) and "remaining" y^r^i). 
The number of yi + i(ri + i) and "remaining" yi(ri + i) is given by the slope of 

* (<) (("»«)-))■ Hence 

K°((m a ) Sn ) = [\^\(m a ) Sn ))\-a{^\(m a ) Sn )))K l+1 + 

((m a ) Sn )) |) [ (l - a (*« ((m a ),„ )) ) - J i+1 • a (*« ((m a ) a „ )) ] ^ . 

and 

|(ma) s Ji = (|^((mo) a J)|-a(* W ((m a ) s J))n,+i + 

(|#«((m a ) s J)|) [(l - a(#«((m a ) s J)) - • a(«P«((m a ) a J)] n,. 

Finally we have 

^°((m a ) 5 J _ 
|(m Q ) s „|i 

a(«pW((rn a ) a J)n i+1 + [(l-a(*W((m a ) 4 J)) - • a(*W((m a ) 4 J)]n i " 



146 7 Optimal routing in two deterministic queues 

Note that the number \(m a ) Bn | depending on n, when n — ► oo, \(m a ) Bn | — ► 
oo and also |(m a ) il „ |i — ► oo. Lemma 32 give 

lim a(#W((m a ) s J) = lim a(«pW((OT a ) n )) = 1 - a, . 

n — >oo n— >oo 

This implies 

j. -^"(("OO _ (1 - aj)K i+1 + [oj - (1 - Qj)] Kj 
n^oc \(jn a ) Sn |i (1 - Qi)TJ i+ i + [a* - k+i (1 - aj)]nj 

Since |^ + i(r i+ i)| is finite then |^ + i(r i+ i)|i and K(y i+1 (r i+ i)) are bounded. 
It comes 

Um K°((m a ) Sn ) |(mq) s Ji = nm K°((m a ) Sn ) 



|(m a ) s „|i |(m a ) s „|i + |^+i(a)|i |(m a ) s „|i 
Um K°(jm a ).J | K°(yi + i(a)) = lim if((m a )„) 
n-oo |(m a ) a „|i |(m a ) s „|i \{m a ) s Ji 

Using Equation 7.23 and Equation 7.24 we obtain 

(1 - oQjfj+i + [Qj - (1 - oQ] ^ < g°((m a )„) 
(1 - Qi)TJ i+ i + [aj - Zj+i (1 - Qi)] rii ~ ™^oo |(m a )„|i 

(1 - Qi)ifi+i + [aj - Zj+i (1 - Qi)] Ki 



(1 - aj)n i+ i + [ati - l i+ i (1 - a*)] 



This finishes the proof of the formulas. 

When a < Ti, by Lemma 29 the minimum number of consecutive in 
m a is larger than Zi . Therefore the inter-arrival times of m a are larger than 
Zi + 1, hence none of the customers have to wait and Ws{m a ) = 0. 



7.8 Appendix: proof of Lemma 37 



We have 



K 



i+l 



m x A'i + i - n i+ i x if. 



rii+i rii rii x rii+i 

Since Vi > 1, x 71^+1 > we have to focus on the sign of riiKi + i — ni + \Ki. 
Using Equations 7.14 and 7.16 yields 



niK i+1 -n i+ iK., 
= n 



(Zi+i + l)ifi + (niZ i+ i-ni_i) rf- +m(l i+1 - - ifi-i 



(Zi+i + l)Ki - Ki_i + (nik +1 - nj_i) rf- + 1 - lj-y^d? 



= +(ni-i)ifj - ni(k + i + l)Ki 



(riiZi+i - TJi_i)rf- +rii(Zi + i - 
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reordering we obtain 

UiKi+i-m+iKi 

= (m^Ki - mKi-!) + nUh+i - + 1j y<%) 

+ni(ni - ni-i)d\ .niK i+1 -n i+ iKi (7.25) 

(7.26) 

On the first hand, by Lemma 34, we know that d\ is strictly positive and d\ +1 
is non negative and that d\ and d\ +1 are non positive. On the other hand 
Equation 7.11 gives us d] + (k+i - = d\ +1 and d] + k+id? = <P i+1 . This 
implies V/j+i > 2, 



That means V/j+i > 1, 

- l)(d] + ^di +1 ) > 0. 



,1 k+l ,2 



Since K2 — niK\ = K2, since K2 = h (d\ + ^ 2 2 ^ d'fj and since d\ + 
(i2 2 > 4 > 0, then JsT 2 > 0, and by induction we have Vi > : 

Tiiifi+i - rii+i^ > . (7.27) 



Therefore Vi > 0, 



When i + 1 is the last step of the expansion the proof holds replacing Z; + i by 
k+i — 1- 
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i) Monotonicity. First we will show by induction on i that the derivative 
function is non positive. 

Step 1. By Equations 7.3 we get : cti = a -1 — Zi, hence 

aa 

Step i. By Equations (7.3) we get = (1 — a^-i) -1 — Zi, therefore 

50 ' >0, (7.28) 



and using the induction hypothesis 
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dai dai da>i-i 



da doti-i da 
Second, using Equations 7.17 and 7.27, we have 
dWs(m a ) - (mK i+ i - n i+ iK 



< 0. 



dai 



Since 



then 



[(1 - Qi) Tl i+1 + (cti - h+i (1 - Qi)) llif 

dW s {m a ) dW s {m a ) dai 



< 0. 



(7.29) 



(7.30) 



da da 

dWs(m a ) 



da ' 



da 



> 0. 



ii) Concavity. Using formulas 7.17 and 7.30 it comes 

d 2 Ws{m a ) 2 {niK i+1 - n i+1 Ki) (-n i+1 + m(l - h +1 )) 



daf 
Hence we get 



[(1 - ai) n i+1 + (ai - (1 - c^)) ni] c 



d 2 W s (m a ) _ d 2 ai dW s (m a ) d 2 W s (m a ) (da 



da 2 da 2 
is equivalent to 

d 2 Ws(m a ) dW s (m a ) 



dai 



+ 



da 2 



da 



da 2 



dai 



2n^ {^Y d 2 a t 
(1 - ai) (m - rii-i) + ami da 2 



(7.31) 



Introducing dai-i and using Formulas 7.3 and 7.14 yield after straightforward 
computations 



2m-\ / daA 2 _ d 2 aj _ 

1 - Qj)(nj - rii-i) + ami \da J da 2 



1 



(l-ev*-i) 2 
At step one we have 
2n 



2m-2 (rii-i - rii-2) f deti-i 
(1 - ai_i) + ai-irii-i \ da 



d 2 ai-x 



da 2 



(7.32) 



( dai 



d 2 



a 1 



da 2 



(1-a) 3 



(1 — Qi)(tji — no) + a\n\ \ da 
Since for all i > 0, (1 — ai) > 0, it follows from 7.30 and 7.31 

d 2 W s (m a ) 



< 0. 



da 2 



< 0. 
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iii) Linearity. The function aWs(m a ) can be computed by replacing m by 
qi in Formula 7.17. Since the recursive computations of the ft's are identical 
for alH > 2 to the recursive computations of the m's, then the Formula 7.32 
still holds 



2ft 



(1 - ati)(qi - qi-i) + onqi \da J da 



doti\' 



d 2 oa 



1 



(l-^-i) 2 



2q. 



i-2 



dai-i 
da 



(1 - ai_i) (qi-i - qi- 2 ) + «i-ift-i 



d 2 aj-i 
da 2 



. (7.33) 



From 



it comes 



2qo ( dai 

(1 - ai)(qi - q ) \ da 

d 2 aW s (m a 



d 2 ai 

da 2 



= 0, 



da 2 



= 0. 



Finally, note that the linearity of the function aWs(jn a ) can also be 
proved directly using Little's law and Farey's intervals as done in [73]. 



Part III 



Several extensions 



153 



This part is dedicated to several extensions of the theory developped in 
the previous parts. 

The first extension concerns a system which is not (max, plus) linear but 
for which the waiting time is still multimodular. This shows that multimodu- 
larity is not limited to the class of systems with a (max,plus) linear dynamic 
and can also be used in systems with losses or dynamic routing. 

The second extension shows a model where the control is on the service 
process and not on the arrival process. This is often called a polling system. 
By using the duality between the polling system and the routing system, we 
show that a polling policy based on sturmian sequences is optimal. 

The last extension shows that multimodularity is also useful to derive 
qualitative properties of closed-loop control and not only for open-loop con- 
trol. In particular, we show monotonicity of the optimal decisions with respect 
to the state of the system. 



8 Networks with no buffers 



8.1 Introduction 

We begin by considering in this Chapter two types of problems: an admission 
into a single server with no buffer, and then the extension of this setting to a 
routing problem. We consider exponential service times and general station- 
ary arrival processes, which include, in particular, the interrupted Poisson 
process, Markov modulated Poisson Process (MMPP) and Markov arrival 
process (MAP) . In spite of the generality of the input process, we are able to 
obtain explicit expressions for the expected cost as a function of the policy. 
This allows us to derive the multimodularity of two types of cost functions 
and to conclude that "bracket" policies are optimal as in [1, 2]. 

Using ideas from [103] we then solve the dual problem [76] which is con- 
cerned with the sharing of a single communication link between multiple 
sources. This is modeled as an optimal assignment problem of a single server 
to several single buffer queues to which packets arrive according to Poisson 
processes. The relation to the previous problem is obtained if we take as 
states in the new problem the vector of free spaces in the buffers (instead of 
the occupancy of the buffers). The distribution of the process of free spaces 
in the new model is the same as the distribution of the process of buffer 
occupancy in the previous problem. This yields the solution of the service 
assignment problem. 

Our original routing problem was also studied recently in [81] (assuming 
however a much simple i.i.d. arrival process). Using the theory of MDPs 
with partial information, he established the periodicity of optimal policies 
and showed for the special case of symmetrical servers and arrival processes 
that the round-robin policy is optimal. For the case of 2 servers, he showed 
that a periodic policy whose period has the form (1, 2, 2, 2, . . . , 2) is optimal, 
where 2 means sending a packet to the faster server. Similar results have been 
obtained in the dual model of server assignment in [42]. The approach in [42, 
81] heavily depend on the Markovian structure of the arrivals. Surprisingly, 
the Markovian approach turns out to be useful also in the non Markovian 
setting (which is due to the general non i.i.d. arrival process that we consider). 
Indeed, we show here that one can use a simple formulation of the problem as 
a Markov decision process (MDP) that allows us to conclude in the general 
setting too, that optimal periodic policies exist. We further relax here the 
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assumption on the distribution of the service times, and allow these to be 
general independent. 

We show in this chapter that policies that are regular in a weaker sense 
are also optimal. More precisely, we establish that the cost function related to 
periodic policies are Schur convex. This allows us to conclude that not only 
regular policies are optimal, but more generally, policies that are regular in 
the majorization sense. We further show that a policy that majorizes a second 
policy has a smaller cost. 

We conclude this chapter with an application of our framework to the 
problem of robot scheduling for web search engines. 

8.1.1 Organization of the chapter 

The Chapter is structured as follows. In the next section we study the ad- 
mission into a single server, and obtain the optimal policy. We then establish 
in the following Section properties of the cost and an ordering between the 
performance of different policies. Section 8.5 deals with the optimal routing 
problem. The service assignment problem is solved in section 8.6 The tech- 
nical arguments that establish the multimodularity properties are delayed to 
Section 8.7. In Section 8.8 we present the MDP formulation of the problem 
and obtain the optimality of periodic policies . We then study in Section 
8.9 the multimodularity of the global cost when considering routing into two 
servers. We conclude with an application to a robot scheduling problem in 
the Web is discussed in Section 8.11. 

8.2 The admission into a single server 

Consider a single server with no waiting room. Let T n be the point process 
representing the arrival epochs, and assume that service times are i.i.d., inde- 
pendent of the arrival process, and exponentially distributed with parameter 
fi. We assume that 6 n := T n+ i — T n is a stationary process (in n). An ar- 
rival can be rejected or accepted by an admission controller. An admission 
policy is a sequence a = (0,1,0,2,—) where Oi = 1 means acceptance of the 
ith arrival, and Oi = means its rejection. The actions are taken without 
any knowledge of the state of the buffer, and if it is already full when the 
packet is admitted, then the packet is lost. Let X n = X n (a) be the number 
of packets in the system just after the nth action is taken. 

We consider first the following problem: 
(PI) Maximize 



where E° is the conditional expectation given a. The policy a is subject to 
the constraint that a fraction of no more than a fraction p of the packets is 
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accepted: 

I 71 

lim — > a n < p. (8.1) 
twoo n — ' 

i=i 

The reason we consider this reward function for maximization is that 
maximizing the average number of packets in steady state is related to 

- maximizing the throughput (the departure rate), and to 

- minimizing the losses (due to both rejection by the controller and to the 
blocking) . 

Indeed, every customer has a sojourn time which is exponentially distributed 
with average pr 1 , and therefore the average actual throughput is equal to 
the product of the long-run average number of packets in the system and p. 
The second equivalence follows since the loss rate is the initial given input 
rate (before the admission control and losses) minus the actual throughput. 

Note that the queue size that we maximize in this section is that ob- 
tained by averaging over the times T n . In some cases this will indeed coincide 
with the time average queue length (for example, if the interarrival times 6 n 
exponentially distributed). In the case it does not coincide, we propose an 
alternative approach in Section 8.4. 

Another motivation to study this problem follows from an interesting 
comparison with the infinite queue system. Below we show that for the current 
system the queue length is maximized by assignment sequences as regular 
as possible; for the infinite buffer system queue lengths are minimized by 
bracket sequences (Chapter 6). Some intuitive understanding can help to 
explain this at first sight contradictory phenomenon: in an infinite queue 
system minimizing queue lengths means minimizing waiting by spreading 
out arrivals; minimizing queue lengths in a system without buffers means 
maximizing losses by making arrivals bursty. 

Before stating the main result, we present a simple coupling property: 

Lemma 45. Fix an arbitrary policy a. Then one can construct a probability 
space (f2,T,F) such that 

(i) the state trajectories starting from different initial states of the server 
couple in a time which is finite w.p. 1, 

(ii) The following holds: 

lim \E[X n (a)\X = 1] - ¥[X n (a)\X = 0]| = 0. 



Proof. We let the interarrival times be the same in both systems. If the policy 
a never accepts packets or if arrivals never occur, then coupling occurs once 
the system empties starting at state 1. Otherwise, let T n be the time at which 
the first packet is admitted. Note that the admission occurs in both systems. 
If both systems are empty at time T„, then coupling occurs at that instant 
(we take service times to be the same in both systems from time T n onwards). 
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Otherwise, system 1 (in which the initial state is of a single packet) has 1 
packet at time T„, and system (which is initially empty) has no packets. 
Coupling is obtained again at time T n by assuming that 

(a) service times of all packets admitted after time T n are the same in both 
systems; 

(b) at time T n , the remaining service time of the packet in service in system 
1 equals to the service time of the packet admitted in system at time T n . 

Due to the memoryless property of the exponential distribution and the 
independence assumption on service times, the above coupling is consistent 
with the probability distribution of the original state processes. This estab- 
lishes (i). (ii) follows from the bounded convergence theorem. q 

Theorem 33. Assume that the system is controlled starting from time T\. 
Assume that the inter-arrival times are stationary, and independent of the 
service times. Assume that the service times are i.i.d. exponentially dis- 
tributed. Then for any 6 £ [0,1], the bracket policy with rate p and initial 
phase 6 is optimal. 

Proof. Due to Lemma 45, we may assume without loss of generality that the 
system contains initially one packet at time To. —Wfr ja X n (a) is multimodular 
(we delay the proof of this property to Section 8.7). Here a denotes the 
random process governing the service completions. f n (a) := — Vfr,aX n (a) is 
clearly monotone decreasing in a. It remains to check conditions < 2 > and 
< 3 > from Section 1.3. 

Recall that Condition < 3 > can be formulated as: the functions f n (a) 
must satisfy the following property. 
For any sequence {ofc} 3 a sequence {/3k} such that 
Vfc,TO, fc > to, fk(Pi, ■■■ ,Pk 

This clearly holds in our case where f n (a) = — Eip >(r X n (a) by setting = 1. 
(The precise justification of the above is from property (i) which appears in 
Section 8.7.). 

Condition < 2 > states: 
/ fc (oi, ...,o fc ) > fk-i(a 2 , ...,o fc ), Vfc > 1; 

this holds in our case for —Err ta X n due to our assumptions on the initial 
state. rj 

8.3 Properties of the cost and of policies 

Lemma 46. g(a) = gi'&a) for any policy a, where i? is the one step shift 
operator. 

Proof. This follows from Lemma 45. Indeed, with a = (01,02, ...), we have 
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where o' = f?o = (02, 03, ....). rj 
Define for any policy a 
def 

r n (a) = min{m > such that a„_ m = 1}. 
Lemma 47. For any policy, KX n (a) is given by 

I r„{a) 

EX ra (a) = E exp I — /j ^ 6k 

\ k=l 



Proof. X n {a) = 1 if and only if since time T n _ Tn ^ there has been no de- 
parture. Thus X n {a) = 1 if and only if during a time period of duration 
Y^k= n -r n (a) there are no end of services. Since 6k is a stationary sequence, 
we obtain the result. rj 

Assume that a is periodic with period P. Let a' be some shift of a such 
that a' s = 1. Thus X nP (a) = 1, n = 0, 1, 2, .... 



= <?(«') = p-EE(X fc (a)) 
fc=i 

Let n = 5Zf=i °(fc)- Define 

1(0) = 0; v(j + 1) = min{/ > : a; = 1}, J = 0, n - 1 

Note that 77(71) = P. Define 

di = 77(1), (Zj =7j(j + 1) j = 2,. ..,rc- 1. 

Lemma 48. a (and thus a') is fully determined by the sequence d up to a 
shift, and the cost g can be expressed as a function of d(a) : 

9(a) = i E E^-^H ^(d) (8.2) 
i=l \ j=0 / 

Proof. By Lemma 47, 

n di-l 1 n dj-1 / T ' 1 ,(i)-j( a ) 

7; 



i=l j=0 i=l j=0 \ fc=0 

^ n di—1 / j \ ^ ti di 

= pE E Eex p -^E^ = pEE Eh pW) 



i=l j=0 \ fc=0 / i=l i=l 



□ 
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We conclude that if a is a periodic policy, then any policy obtained 
from a by changing the order of the d sequence that characterizes a, 
achieves the same cost. Thus a non-bracket periodic policy of a period 
(1,0,1,0,1,0,0,1,0,0) has the same cost as a bracket policy of the form 
(1,0,1,0,0,1,0,1,0,0). 

This property can be seen to be a special corollary of a more general result 
that will be presented in Theorem 34 below. 

Consider two n-dimensional vectors of integers d(l),d(2). 

Definition 14. (Majorization [88]) 

We say that d(2) majorizes d(l), which we denote by d(l) -< d(2), if 

' k k 

I> w (l) < ^d [ , 1 (2), fc=l,...,rc-l, 
i=i i=i 

< 

n 7i 

X)dw(i) = E d w( 2 ) 

. i=l i=l 

where d^{j) is a permutation of di(j) satisfying rf[i](j) > d[ 2 ]{j) > ••• > 
d[ n ](j), j = 1,2. 

A function / : ffi™ -> M is Schur convex if d(l) -< d{2) implies f{d{l)) < 
f(d(2)). f is Schur concave is d(l) -< d(2) implies f(d(l)) > f(d(2)). 

Lemma 49. g'{d), defined in (8.2) is Schur concave. 

Proof, g' can be written as the sum 

s'(d) = -lf>(d0) 

i=l 

where ip is the term in brackets in eq. (8.2). Since 

ipi(m + 1) - i)i{m) = Ee-v E ™=<> 6k 

is monotone decreasing in m, it follows that ipi are concave in di. The proof 
is then established by using proposition C.l on p. 64 in [88]. □ 

Theorem 34. Assume that a and a 1 are two periodic policies with the same 
period P and the same sum 

p p 
i=i i=i 

Ifd(a') -< d(a) then g(a') > g(a). 
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Proof. Follows from the Schur concavity of g, see [88] p. 54. j-j 

The above theorem allows us to "improve" any given periodic policy by 
replacing it by a more "regular" one, where by more regular we mean a policy 
whose distance sequences are majorized by the less regular one. 

A similar result was obtained in [42] in a related model (which we discuss 
in Section 8.11 under the assumption that the sequence 6 n is i.i.d.). 

8.4 Time averages 

In the previous sections we took averages of the costs as seen at times T„, 
i.e. at arrival times. The problem with this is, however, that T n are in fact 
the times of potential rather than actual arrivals. In practice, arrivals occur 
only at a subsequence of T n which depend on the policy. In this section we 
obtain similar results for the actual arrival process. 

We consider the same model of the system as well as the statistical as- 
sumptions as in Section 8.2. 

Instead of describing a policy using a sequence a, it will be more helpful to 
consider an equivalent description using the distance sequence d = (di, d,2, ...). 
Define D(n) = Y^k=o The ac t ua l arrivals occur at times T D ^ n ),n £ N. We 
define the process £ ra to be the number of packets in the buffer just prior to 
time T D ( n y (If we took, as in the previous sections the time after a decision, 
then the number of packets would always be 1. 

The motivation for considering the system at arrival instants is the fol- 
lowing. Whenever an actual arrival finds a packet in the system there is a 
loss. Thus minimizing the average number of packets at actual arrival times 
will also minimize the fraction of losses and maximize the throughput. 

We consider first the following problem: 
(Ql) Minimize 



subject to the constraint that a fraction of no more than a fraction p of the 
packets is accepted, or in other words, 



Recall that a policy d is bracket with rate 1/p if and only if its related 
policy a is bracket with rate p, see Lemma 21. 

Theorem 35. Assume that the system is controlled starting from time T\. 
Assume that the inter- arrival times 6 n are stationary and independent of 
the service times. Assume that the service times are i.i.d. exponentially dis- 
tributed. Then for any 6 £ [0, 1], the bracket policy with rate 1/p and initial 
phase 6 is optimal. 




,71 




(8.3) 
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The proof of the Theorem is similar to that of Theorem 33. The multi- 
modularity of E£„(d) is established in Section 8.7. 

We now present properties of the cost for periodic policies, which are 
the analogous of Section 8.3. We first define the expected average cost for a 
periodic sequence d = (di,...,dp) given by 

i p 

G(d) = -£E(-M) (8.4) 

i=l 

when the interarrival times are i.i.d. Since this is a sum of functions that are 
convex in the rfj's, it is Schur convex in d (proposition C.l on p. 64 in [88]). 
We thus conclude the following (see [88] p. 54): 

Theorem 36. Assume that d and d! are two periodic policies with period P 
and the same sum: 

p p 

n = ^di = Y^ d 'i- 

i=i i=i 

Ifd'<d then G(d') < G(d). 



8.5 Routing to several servers 

We now consider K servers (all with a single buffer), fed by a stationary 
input process as in Section 8.2. We make the same probabilistic assumptions 
as before on the service times in each server. Let be the number of packets 
(0 or 1) being served by server i at the nth time epoch. A routing policy is 
a sequence a = (ai,02, ...) where en = j means routing of the ith arrival to 
queue j. We consider the following routing problem: 
(P2) Maximize 

g(a) 4l f lim 

n — >oo rh . . 

3 = 1 2=1 

where hi are some given positive constants. 

The following theorems are the result of the properties we established in 
the previous section, the multimodularity properties (which is established in 
the next section) as well as the results in Chapters 1 and 4. 

Theorem 37. Consider the symmetric system, i.e., fii = ■ ■ ■ = [ik- Then 
the round robin policy maximizes g(a) (and hence, the expected average 
throughput). 

Theorem 38. Consider the case of two servers. Then there is some p* such 
that for any 6, the policy that routes packets to server 1 according to the 
bracket policy with rate p* and initial phase 6, and otherwise routes packets 
to server 2, is optimal. 
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Theorem 39. Consider the case of two types of servers: a setK\ C {1, ...,K} 
of servers with fi = p,\, and the remaining set K 2 of servers with \x = \x 2 . 
Assume that hi = h 1 are the same for all i £ K\ and that hi = h 2 for all 
i e K 2 . Then there exists some p* such that that for any 6, the following 
policy is optimal: 

it routes packets to the 1st group server according to the bracket policy with 
rate p* and initial phase 6, and otherwise routes packets to the second group 
of servers. Within each group of servers, the order of service is round-robin. 

The proof of all three theorems is based on Proposition 11, which states 
that if a tuple (pi , p 2 , ■ ■ ■ , Pk ) is made of less than (or of exactly) two distinct 
numbers, then it is balanceable. In other words, there exist a policy a such 
that for each i = 1,...,K, the sequence of indices in o that correspond to 
routing arrivals to queue i are is a bracket with rate pi. The optimality of 
balanceable sequences was established in Chapter 1. 

Remark 13. For any a, let a 1 = {a l n , n € N} be the binary sequence such that 
a' n = 1 if and only if a n = i. Using the result of Section 8.3, and in particular 
the Schur convexity of g as a function of the cfs (see Lemma 49), one can 
show that if there are two periodic policies a and b with the same period P, 
such that for any i = 1, K, 

- d{b l ) -< dia*), 
then g(b) > g(a). 



8.6 The service assignment problem 

Consider K Poisson processes with parameters yui, /j,k respectively, of 
packet arrivals into K respective single buffer queues. One buffer can obtain 
transmission opportunity at a time. Let T n be time at which the nth trans- 
mission opportunity occurs. If an arrival occurs to a buffer that is full then 
it is lost; if the buffer is empty then the arriving packet is stored till a trans- 
mission opportunity to that buffer arrives. If a buffer receives a transmission 
opportunity at time T n and it has a packet then this packet is transmitted, 
and immediately after time T n , a new arriving packet can be stored in this 
buffer. If there is no packet in the buffer then this transmission opportunity 
is lost. We assume that 6 n := T n+ i — T n is a stationary process (in n) and 
independent on the arrival process. 

The role of the controller is to decide to which buffer will the next trans- 
mission opportunity be assigned. A service assignment policy is a sequence 
a = (01,02,...) where Ui = j means that the ith transmission opportunity 
will be to queue j. We assume that the controller has no information about 
the buffers' contents. 
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Let Z l n be the number of packets (0 or 1) at buffer i just after the nth 
action is taken. Denote Y£ = 1 — Z % n . It thus corresponds to the 'vacancies' 
process, as Y£ equals one if the ith buffer is empty just after the nth action 
is taken (i.e. after time T n ). We consider the following problem: 
(P3) Minimize 



where hi are some given positive constants. 

The above objective corresponds to the minimization of blocking proba- 
bilities, since blocking occurs at queue i between T n and T„+i if and only if 



Note that by minimizing blocking probabilities, we maximize the through- 
put. 

We now make the following key observation. Fix an arbitrary sequence a. 
Then the distribution of the vacancies process {Y^}„,i in the service assign- 
ment problem is the same as the distribution of the buffer contents process 
{Xn} n ,i in the routing problem. Hence, using the results of the previous 
section, we get the following main results. 

Theorem 40. Consider the symmetric system, i.e., fii = ■ ■ ■ = [Ik- Then 
the round robin policy minimizes g(a) (and hence, maximizes the expected 
average throughput). 

Theorem 41. Consider the case of two servers. Then there is some p* such 
that for any 6, the policy that assigns transmission opportunities to buffer 1 
according to the bracket policy with rate p* and initial phase 6, and otherwise 
assigns transmission opportunities to server 2, is optimal. 

Similarly, one obtains the dual of Theorem 39. 
8.7 The multimodularity 

Lemma 50. The two following statements are true, (i) X n (a)) is mul- 
timodular, 

(ii) E a £ n (d) is multimodular. 

def 

Proof, (i) The proof is based on the following useful properties. Let q(a) = 
maxjra < n such that a m = 1} . Consider two policies a, a'. Then, 



Property (A): if q(a) = q(a') then X n (a) = st X n (a') (this is equivalent to 
K (X n (a)) =E a (X n (a')) and to ¥(X n (a) = 1) = ¥(X n (a') = 1)). 




1. 



This is due to the memoryless property of the exponential service times. 
Thus we can replace our original system by one where at each acceptance, 
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the new packet replaces the one in service, instead of being rejected; the 
distribution of the process X n will not change. 

Property (B): if q(a) < q(a') then X n (a) < s t X n (a') (which is equivalent 
toK(X n (a))<K(X n (a'))). 
This is obtained by a similar argument. 

We shall now check relation (1.1) in the definition of multimodularity 
(Chapter 1) for any a such that a, a + w,a + v,a + w + v are feasible. 

Let v = e„. We have for any w £ T , w ^ v, 

X n (a + w + v) = X n (a + v) = X n (v) = 1, 

and 

X n (a + w) < st X n (a). 

The first relation follows from property (A) above, and the second from prop- 
erty (B), since 

q(a + Si) < q(a), i = 2, n and q(a — e^) < q(a) 

(recall that Si corresponds to shifting an arrival to the past). This implies 
that 

E a (-X n (a + w +v)) = E a (-X n (a + v)), 
K(-X n (a + w)) >E a (-X n (a)). 

Hence relation (1.1) is satisfied. 

Next, assume v = —e\. We consider io^ii (and thus restrict to n > 2). 
For a + v to be feasible, we must have oi = 1. For a + w to be feasible, we 
have w ^ S2, and q(a) > 1. It then follows from property (A) that 

X n (a) = st X n (a + v), X n (a + w) = st X n (a + v + w) . 

Hence relation (1.1) holds with equality. 

It remains to check v = Si, Sj = w, with j > i. Since a + Sj is feasible, it 
follows that q(a) > j. Hence, by property (A), 

X n (a + Si) = st X n (a). 

Similarly, q(a + Sj) > j — 1; since it is feasible then Oj_i = 0, so that i < j — 1 
(in order for a+Si to be feasible, we have to exclude i = j — 1, since Oj_i = 0). 
Hence 

X n (a + Si + sj) = st X n (a + sj). 

Hence relation (1.1) holds with equality, which concludes the proof of (i). 

(ii) Let v be any one of the vectors in the set {— ei, S2, s n -\}. Then 
due to Property (A), for any w ^ v, 

£n{a) = st i n {a + v), £ n {a + w) = st £ n {a + v + w). 
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Hence relation (1.1) is satisfied. By symmetry it holds for any w in the set 
{— ei, S2, It thus remains to check the case v = e n ,w = s n . 
From Lemma 47 we have: 

E£ n (d) =Eexp (-vJLh^J ■ 

Let x n = m, and let denote 

2/ = exp ^-^X^fcj • 

Then 

E£(x + v + w) = E^(x) = Eye"^ m + 1 = Eye"^ 1 
E£(x + w) = Ey. 

Since the function /(x) := ye^^ x is convex in x, it follows that f{6\ +z)—f(z) 
is increasing in z, so that 

f(6i + 6m+i) - /(W) > - /(0). 

By taking expectations, this implies relation (1.1) for E^„ (a), which concludes 
the proof. □ 



8.8 MDP formulation and new structural results 

We reconsider our routing problem into K parallel servers with no waiting 
room in the framework of Markov Decision Processes. Recall that packets 
arrive at times (T„)„ £ n. We use the convention that T\ = 0. and we assume 
that the process (6 n )„ e m of interarrival times is stationary. Upon arrival of 
a packet a controller must route this packet to one of the K servers in the 
system. 

We allow in this section for more general service times: the service time 
has a general service distribution G m when routed to server m e {1, . . . , K}. 
If there is still a packet present at the server, where the packet is routed 
to, then the packet in service is lost (we call this the preemptive discipline). 
In the special case of an exponential distribution, one can consider instead 
the non-preemptive discipline in which the arriving packet is lost; the results 
below will still hold. 

We assume that the controller, who wishes to minimize some cost func- 
tion, has no information on the state of the servers (busy or idle). The only 
information which the controller possesses is its own previous actions, i.e. to 
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which server previous packets were routed. We assume that this informa- 
tion does not include the actual time that has elapsed since a packet was 
last routed to that server. This assumption enables us to study the embed- 
ded Markov chain of the continuous-time process; i.e. we can consider the 
discrete-time process embedded at the epochs when packets arrive. Now the 
mathematical formulation of this problem is given by the following T- horizon 
Markov decision process. 

Let X = (NU{oo}) be the state space. The m th component x m of x £ X 
denotes the number of arrivals since a packet was last routed to server m. 
Let A = {1, . . . , K} be the action space, where action a € A means that a 
packet is routed to server a. Since actions are taken before state transitions, 
we have the following transition probabilities: 



Define the immediate costs by c t (x,a) = f a (x a ), which reflects that the costs 
only depend on the chosen server and the state of that server for t < T. The 
terminal costs are given by 



(note that the terminal costs use the same functions f a ). Defining these ter- 
minal costs will be essential for the mathematical results, as will be illustrated 
later in Example 9. It has also a natural physical interpretation, as will be 
illustrated in Remark 16. 

The set of histories at epoch t of this Markov decision process is defined as 
the set Tit = {X x A) 1 ^ 1 x X. A policy n is a set of decision rules (tti,^, ■ ■ .) 
with 7Tt : Tit — * A. For each fixed policy ir and each realization ht of a 
history, the variable At is given by At = irt(ht). The variable Xt+i takes 
values x t +i £ X with probability 1 p{x t +i \x t ,a t ). With these definitions the 
expected average cost criterion function C(tt) is defined by 



where x = {x\ , . . . , xk) is the initial state. Let II denote the set of all policies. 
The Markov decision problem is to find a policy n* , if it exists, such that 
C(tt*) = min{C(7r) | tt € II}. This Markov decision model is characterized by 
the tuple (X, A,p, c). 



1 In the general MDPs, the variables At and Xt are random; here they are deter- 
ministic since the transition probabilities take only values of O's or l's 





(8.5) 
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Example 6. Suppose that the controller wishes to minimize the number of 
lost packets (i.e. the number of preempted packets) per unit time. The cost 
function f m (n) will typically be a decreasing function in n, because a longer 
time interval between an assignment to server to results in a smaller proba- 
bility that a packet, that was previously assigned there, is still in service. 

Assume that the arrival process is a Poisson process with rate A and that 
services at server i are exponentially distributed with rate /ii independent 
of the other servers. Let Si be a random variable which is exponentially 
distributed with rate Then f m {n) = V(S m > 8\ + ■ ■ ■ + 6 n ) = [A/(A + 

In the setting described in Example 6 we obtained a decreasing cost func- 
tion f m (x). In Section 8.10 we discuss this application in more detail. In Sec- 
tion 8.11 we describe another application of our model, where the obtained 
cost function is increasing. In this section, we make some general assumptions 
on the cost function in order to cover all these different structures. Moreover 
we investigate structural properties of the optimal policy. 

Assume that all the f m are convex and that one of Conditions (8.6)-(8.8) 
defined below holds: 

lim (f m (x + 1) - f m (x)) = oo m = 1, ...,K. (8.6) 



f m are strictly convex and lim (f m (x) — a m x) = C, m = 1, . . . , K 

x—>oo 

(8.7) 

(by strictly convex we mean that for all x, f m {x + 2) — f m (x + 1) > f m {x + 
!)-/(*))■ 

f m (x) = a m x + C, m = l,...,K, (8.8) 

where a m > and C are constants (and C does not depend on to). Note 
that Condition (8.8) is not included in (8.7): it violates its first part. Con- 
dition (8.6) covers the case where f m grows more than linearly, whereas 
(8.7)-(8.8) cover the case where f m grows asymptotically linearly. These 
conditions are complementary to (8.6) since any one of them implies that 
lim^oo (f m (x + 1) - f m (x)) < oo. In Conditions (8.6) and (8.7), f m is 
strictly convex. 

Theorem 42. Assume that one of Conditions (8.6)-(8.8) holds. Then 

(i) There exists an optimal policy that uses every server infinitely many 
times, thus supjj | wj = to} = oo for to € {1, ... , K}. 

(ii) There exists an periodic optimal policy. 

Before proving the theorem we note that there are certain ways in which the 
conditions (8.7)-(8.8) cannot be relaxed. We illustrate this below with cost 
functions for which the above theorem does not hold. 
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Example 7. Consider the case of two servers with the costs 

fi(x) = ciiX + bi exp(-£i x) +Ci, i = 1, 2, 

where c\ < c 2 , and where b\ > and li > are some constants (as follows 
from the next remark, the sign of cii is not important) . Then for a sufficiently 
small value of b\ , the policy that always routes to server 1 is the only optimal 
policy for any finite horizon N. 

Indeed, assume first 6^ = for all i and let u be any policy that routes at 
its n th step to server 2. By changing the action at this step into an assignment 
to server 1 we gain c 2 — c\. 

By continuity of the cost in the bi's, we also gain a positive amount using 
this modified policy if bi ^ provided the bi's are sufficiently small. Hence 
for bi sufficiently small, a policy cannot be optimal if it does not always route 
to server 1. 

When using the average cost, the cost is not affected anymore by any 
changes in the actions provided that the frequency of such changes converges 
to zero. Hence for the average cost, there may be other policies that are 
optimal, but still, any policy for which the fraction of customers routed to 
server 2 does not converge to cannot be optimal. 

We conclude that we cannot relax (8.7) or (8.8) and replace C by Cm- 



Remark 14. Note that when the cost f%{x) contains a linear term cax then 
the total accumulated cost that corresponds to the term (nx over any horizon 
of length N is Oi(N + Xi), where x = (0:1,0:2) is the initial state. This part 
does not depend on the policy. If we use a policy 7r and then modify it by 
changing an assignment at time t < N from server i to server j ^ i then 
the linear part of the cost at time t under the modified policy decreases by 
di Xi(t) — cij xj(t), but it then increases by the same amount at time t + 1. 
Thus the accumulated linear cost is independent of the policy. (Note that 
this argument is valid due to the definition of the cost at time N in (8.5). 



Example 8. Consider the case of two servers with the costs fi(x) = a\x and 
fi(x) = exp(— £ 2 x). For any finite horizon N and for £ 2 > 0, the only optimal 
policy is the one that always routes to server 1. Note that the average cost 
until time N of the policy that always routes to server 1 is 

Nh(l)+h(N) _ l2 

N -e +ai. 

The average cost of the policy that always routes to server 2 is 

JV/i(l)+/ 2 (JV) ^ e-'> N 
N =ai + ^V^- 
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Again, for the average cost there are other optimal policies but they have to 
satisfy the following: the fraction of customers routed to queue 2 by time N 
should converge to zero as N — ► oo. 

This illustrates the necessity of the first part of Condition (8.7). For £ 2 < 
0, the only optimal policy is the one that always routes to server 2. 

Next we present an example to illustrate the importance of the terminal cost. 

Example 9. Assume that there are two servers, and that the costs are given 
by fi{x) = x 2 and f2{x) = 2x 2 . Assume that the terminal costs cn{x) were 
zero, then the policy that always routes to server 1 is optimal. 

Proof (Proof of Theorem 42). First suppose the cost function satisfies 
Condition (8.8). Interchanging assignments between any two servers for any 
finite horizon does not result in changes in cost for that horizon, due to the 
linearity of the cost function and the terminal cost. Hence any periodic policy 
that routes packets to all servers is optimal. 

We consider next Conditions (8.6) and (8.7). Instead of describing a policy 
using a sequence it, we use an equivalent description using time distances 
between packets routed to each server. More precisely, given an initial state 
x, define the j th instant at which a packet is routed to server m by 

V m (0) = -x m 

and rj m (j) = minji | max (r) m (j - 1), 0) <i < N and 7T; = to}, 

for j E N (the minimum of an empty set is taken to be infinity.) Define 
the distance sequence d(m) by d(m) = (di(m), ^(to), ...), with dj(m) = 
V m (j) ~ V m (j ~ l)j f° r j G N. (For simplicity we do not include the N in the 
notation.) 

Let 7r be an arbitrary policy and m be an arbitrary server (fixed from now 

def 

on). Assume that the distance sequence d = d(m) for this server has the 
property that lim sup^^^jdj \j £ N} = oo. We shall construct a policy tt' 
with distance sequence d v such that limsupjY^^jdJ \j £ N} is finite and 
C(tt') < C(tt). 

Assume first that / satisfies Condition (8.6). Choose no such that for all 

n > no 



for all /. Since the supremum of the distance sequence is infinity, there is 
a j (and N) such that dj > n + 2K + 1. Consider the 2K + 1 consecutive 




> 



max 

l<fc<2if+l 



[fi{2K + 1) - /,(*) - fi(2K + 1 - *)) 
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assignments starting at rj m (j — 1). Since there are K servers, it follows that 
there is at least one server, to which a packet is assigned three times during 
that period, say to'. Denote the distance (or interarrival) times of those three 
assignments to to' by di(m') and d 2 (m'). Replace the second assignment to 
to' in this sequence by an assignment to server to. Denote the new distance 
(or interarrival) times to m by d'- and d"- (if rj(j) = N then the distance d"- is 
not a real interarrival time) . Consider the cost for a horizon of length / where 
I is an arbitrary integer larger than r\(j — 1) + no + 2K + 1. 

[f m (dj) + f m '(di(m')) + / ra -(d 2 (m'))] - 
[fmid'j) + fm(d") + f m -(di(m') + Mm'))] 
= [f m (dj) - / m «) - / m (4)] - 

[f m >(di(m')+d 2 (m')) - f m .(di(m')) - f m >(d 2 (m'))] > 0, 

where the last inequality follows from the choice of no ■ 
Consider now Condition (8.7). Since by assumption the supremum of the 
distance sequence di = di(m), I = 1,2, ... is infinity, there is a j (and N) such 
that dj > 2n + 2K + 1, for some n. Let p := min{/ m (fc) +a m — f m {k + l) \ m = 
1, . . . , K, k = 1, . . . , K}. Note that p is positive, since Condition (8.7) implies 
that (fi{x) — a\x — C) is positive and strictly decreasing (for all I). Now choose 
n such that 2q = 2(/ m (n) — a m n — C) < p. Note that this is possible, since 
fm{n) — a m n — C goes to zero as n goes to infinity. Consider the 2K + 1 
consecutive assignments starting n units after n(j — 1). There is at least one 
server, to which a packet is assigned three times, say m' . Replace the second 
assignment to m' in this sequence by an assignment to server m. 

Define the function gi(k) = fi(k) — aik — C for all i and consider the 
cost for a horizon of length I where / is an arbitrary integer larger than 
rj(j — 1) + 2n + 2K + 1. The decrease in cost due to the interchange is 

[fmidj) + f m '(di(m')) + f m ,(d 2 (m'))] - 
[fm(d' 3 ) + fm(d'j) + f m -(di(m') + d 2 (m'))] 
= [dm{dj) + g m '{di{m')) + g m '{d 2 {m'))] - 
[gmid'j) + 9m{dj) + 9m'{di{m') + d 2 {m'))] 

> [9m{dj) + g m '{di{m')) + gm'(d 2 (m'))] - [2g m {n) +g m '{di{m') +1)] 
= 9m{dj) + g m '{d 2 {m')) + [g m '{di{m')) - g m '{di{m') + 1)] - [2g m (n)] 

> 9m{dj) + g m '{d 2 {m')) + p- 2q > 0, 

where di(m'), d 2 (m'), d'- and d"- are defined as before. The first inequality 
follows from the fact that n < d'-, n < d"-, di(m') + 1 < di(m') + d 2 (m') 
and fm{%) — d m x — C is decreasing. The second inequality follows from the 
definition of p. Since f m — a m — C is positive it follows by construction of n 
that the last inequality holds. 
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Repeating the interchange procedure for every j for which dj > 2n+2K+l 
(when dealing with Condition (8.7)) or for which dj > n + 2K + 1 (when 
dealing with Condition (8.6)) provides us a policy n' such that C(tt') < C(tt) 
and sup{(fj | j G N} < 2n + 2K + 1. By repeating this procedure for every 
server, we get an optimal policy that visits a finite number of states. By 
Chapter 8 of Puterman [95] we know that the optimal policy can be chosen 
stationary. It follows that ir n (h n ) = iro(x n ). Since the state transitions are 
deterministic it follows that the optimal policy is periodic. □ 



8.9 Multimodularity of the global cost: two servers 

We consider in this Section the special case of two servers. 

In previous Sections we have already established the multimodularity of 
the cost related to a single server. This was sufficient to get optimality results 
for the multidimensional case (the routing problem) since the optimality of 
bracket policies has been established under the conditions that the cost of 
each component separately be multimodular (Section 1.4). 

In this Section we consider directly the multimodularity of the combined 
cost of the two queues. Although this is not necessary for obtaining the 
regularity structure of optimal policies, this is an important property for 
optimization purposes, as it allows us to use local search procedures (that 
are based on the fact that local minimum is also a global one, see Corollary 
2). 

The notation of the distance sequence can be beneficially used to approach 
the decision problem. After the first assignment to server to, the distance 
sequence d(m) for server to is periodic, say with period P(m). Therefore in 
future discussions we will write ir = (tt\, . . . , 7r„) for the periodic assignment 
sequence with period n and with a slight abuse of notation we denote the 
periodic distance sequence for server to by d(m) = (d(m)i, . . . ,d(m)p^). 

The periodicity reduces the cost function in complexity. Since we use 
the expected average cost function, we only have to consider the costs in- 
curred during one period. It would be interesting to establish multimodular 
properties for any K . Unfortunately it is not clear how even to define multi- 
modularity for K > 2. We thus consider below K = 2. The expected average 
cost is given by 

K 1 K P(m) 

= E S™M = "EE fm(d(m)j). (8.9) 

m=l m=l j=l 

It is tempting to formulate that g m (7r) is multimodular in it for to = 1,2. 
Note that this is not necessarily true, since an operation v £ T applied to tt 
leads to different changes in the distance sequences for the different servers. 

We shall thus use an alternative description for the average cost through 
the period of server 1. Define g' m as follows: 
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1 <*(!) 

<?:» = - E .w^ 1 ))- ( 8 - 10 ) 

3=1 

We note that the function g' m {i^) only looks at the distance sequence assigned 
to the first server with respect to tt using cost function f m . By the symmetry 
between the assignments to the two servers g(ir) can now be expressed as 
g(ir) = g[(Tr) + g' 2 (3 — tt). (3 is the vector whose components are all 3.) Note 
that dj(l) = dj(l)(ir) is a function of tt, and we have 

dj(l)(3-7r)=dj(2)(7r). 

We first prove that g' m {ir) is multimodular in tt. Then multimodularity of 
g(ir) follows as the sum of two multimodular functions. 

Lemma 51. Assume that f m are convex. Let it be a fixed periodic policy with 
period n. Let g' m {Tf) be defined as in (8.10). Then g'^Tt) is a multimodular 
function in tt. 

Proof. Since it is a periodic sequence, the distance sequence d = d(l) is 
also a periodic function, say with period P. Now, define the function hj for 
j = 1, ... ,P by hj(ir) = f m (dj). The function hj represents the cost of the 
(j+l) st assignment to server m by looking at the j th interarrival time. We will 
first check the conditions for multimodularity for V = {&i, . . . , b n -i}, where 
fys are the elements of T (i.e. bo = —ei,h = S2, ■ ■ ■ , b n -i = s n , b n = e„). 

Let v, w E V with v ^ w. If none of these elements changes the length of 
the j th interarrival time then hj(n) = hj(n + v) = hj(n + w) = hj(-7T + v+w). 
Suppose that only one of the elements changes the length of the interarrival 
time, say v, then hj{-K + v) = hj{~K + v + w) and hj(n) = hj{-K + w). In both 
cases the function hj(Tr) satisfies the conditions for multimodularity. 

Now suppose that v adds and w decreases the length of the j th interarrival 
time by one. Then dj(Tr +v) — dj(iv) = dj{it + v +w) — dj{it + w). Since hj is 
a convex function, it follows that hj(ir + w) — hj{~K + v +w) > hj{~K) — hj(ir + 
v). Now the multimodularity condition in Equation (1.1) directly follows by 
rearranging the terms. Since g' m {Tf) is a sum of hj(n) it follows that g' m {Tr) is 
multimodular for V. 

Now consider the elements bo and b n and note that the application of bo 
and b n to tt splits an interarrival period and merges two interarrival periods 
respectively. Therefore 

n 9' m {^ + b o) = ng' m {Tr) - f m {di) - f m {dp) + f m {di + d P ), 

ng'min + b n) = ng'min) ~ fm(dp) + f m (d P - 1) + /m(l), 

ng' m {ir + b +b n ) = ng' m {^) - f m {di) - f m {dp) + f m {di + 1) + f m {d P - 1). 

Now n[g' m {Tr + b ) + g' m {n + b n ) - g' m {ir) - g' m {ir + b + &„)] = [f m {di + 
d P ) + fmiX)] - [fm(di + 1) + f m (d P )]. Let k = di + d P + 1. Since the 
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function f m (x) + f m (y) with x + y = k is a symmetric and convex function, 
it follows from Proposition C2 of Chapter 3 of Marshall and Olkin [88] , that 
fm(x) + f m (y) is also Schur-convex. Since {d\ + l,d P ) -< {d\ + d P ,l), the 
quantity above is non-negative. 

In the case that we use w = bo and »eV such that v does not alter d\, 
then it follows that g' m {^ + v + w) = g' m (ir + v) + g' m (ir + w) — g' m (n). The 
same holds for w = b n and v £ V such that v does not alter dp. Suppose that 
v does alter d\, then we have n[g' m (Tr + bo) + g' m {^ + v) — g' m {Tr) — g' m {^ + 
bo+v)] = [f m (di +d P ) +f m (di -1)]- [f m (di +d P -l) + / m (rfi)]. When 
v alters d P we have n[g' m (ir + b n ) + g' m {-K + v)- g' m (ir) - g' m {-K + b n + v)] = 
[f m {dp + l) + f m {l)\ - [fm{dp) + f m (l + l)] for some I <d P . Now by applying 
the same argument as in the case of bo and b n we derive multimodularity of 
g' m (.Tt) for the base T. □ 

Now we will prove that g(7r), which is given by g(7r) = g'-^) + g' 2 {3 — vr) is 
multimodular. The proof is based on the fact that if a function is multimodu- 
lar with respect to J 7 , then it is also multimodular with respect to Q (defined 
above Definition 1). 

Theorem 43. Let g[ and g' 2 be multimodular functions. Then the function 
g(ir) given by g(ir) = c\ g{(7r) + c 2 g' 2 (3 — 7r) for positive constants c\ and c 2 
is multimodular in it. 

Proof. Let v, w £ T , such that v ^ w. Then 

g(ir+v) + g(n + w) 

= ci g[(ir + v) + c 2 g' 2 {3 - tt - v) + c 1 g[{-n + w) + c 2 g' 2 {3 - tt - w) 
= c 1 [g' 1 {-K + v)+g' 1 {^ + w)] + c 2 [g' 2 (3 - n - v) + g' 2 (3 - n - w)] 
> ci [g[(7r) +g[(7r + v + w)]+ c 2 [g' 2 (3 - tt) + g' 2 (3 - tt - v - w)] 
= ci g[(ir) +c 2 g' 2 {3 - tt) + c 1 g[{it + v + w) + c 2 g' 2 {3 - tt - v - w) 
= g(n) + g(n + v + w). 

The inequality in the fourth line holds, since g[ is multimodular with respect 
to T and g' 2 is multimodular with respect to Q. rj 

8.10 Examples of arrival processes 

In today's information and communication systems the traffic pattern may 
be quite complex, as they may represent a variety of data, such as customer 
phone calls, compressed video frames and other electronic information. Mod- 
ern communication systems are designed to accommodate such a heteroge- 
neous input and therefore the arrival process used in mathematical models 
is of crucial importance to the engineering and performance analysis of these 
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systems. In this section we elaborate on the setting of Example 6 with dif- 
ferent arrival processes and derive explicit formulae for the cost function for 
the corresponding arrival process. 

Assume that the controller wishes to minimize the number of lost pack- 
ets (i.e. the number of preempted packets) per unit time; note that this is 
equivalent to maximizing the throughput of the system. Furthermore let the 
services at server i be exponentially distributed with rate /ii independent of 
the other servers. Since we know that there exists a periodic optimal policy, 
we can write the cost induced by using policy it by 

M P(m) 
m=l j=l 

in case the arrival process is a Poisson process with parameter A. In Koole [81] 
it was shown that the optimal policy has a period of the form (1,2, ... ,2), 
where 2 is the faster server. In Chapter 6 this result was generalized for 
general stationary arrival processes. Hence suppose that A = 1 and [i\ = 1, 
then the cost function can be parameterized by the period n and the server 
speed H2 > Hi given by 

By solving the equations g(n,H2) = g(n + 1,^2) for n > 2 we can compute 
the server rates ^2 for which the optimal policy changes period. For example: 
the optimal policy changes from (1,2) to (1,2,2) when fi 2 > 1 + y/2. The 
results of the computation are depicted in Figure 8.1. 

Markov Modulated Poisson Process 

An important class of models for arrival processes is given by Markov mod- 
ulated models. The key idea behind this concept is to use an explicit notion 
of states of an auxiliary Markov process into the description of the arrival 
process. The Markov process evolves as time passes and its current state mod- 
ulates the probability law of the arrival process. The utility of such arrival 
processes is that they can capture bursty inputs. 

The Markov Modulated Poisson Process (MMPP) is the most commonly 
used Markov modulated model and is constructed by varying the arrival rate 
of a Poisson process according to an underlying continuous time Markov 
process, which is independent of the arrival process. Therefore let {X n \ n > 
0} be a continuous time irreducible Markov process with k states. When the 
Markov process is in state i, arrivals occur according to a Poisson process 
with rate \i. Let pij denote the transition probability to go from state i to 
state j and let Q be the infinitesimal generator of the Markov process. Let 
A = diag(Ai, . . . , A&) be the matrix with the arrival rates on the diagonal 



A 
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Fig. 8.1. Relationship between n and fi2 



and A = (Ai, . . . , \k) the vector of arrival rates. With this notation, we can 
use the matrix analytic approach to derive a formula for f m (n) . 

Theorem 44. (Section 5.3, [92]) The sequence {(X n ,6 n )\n > 0} is a 
Markov renewal sequence with transition probability matrix F(t) given by 

F(t) = f e&- A > duA=[l- e ( Q - A ^] (A - Q)- 1 A 
Jo 

The interpretation of the matrix F(t) is as follows. The elements Fij(t) are 
given by the conditional probabilities V(X n+ i = j, 6 n+ i < t \ X n = i) for 
n > 1. Since F(t) is a transition probability matrix, it follows that F(oo) 
given by (^1 — Q)~ 1 A is a stochastic matrix. 

The MMPP is fully parameterized by specifying the initial probability 
vector q, the infinitesimal generator Q of the Markov process and the vector 
A of arrival rates. Let the row vector s be the steady state vector of the 
Markov process. Then s satisfies the equations sQ = and se = 1, where 
e = (1, . . . , 1). Define the row vector q = sA/sX. Then q is the stationary 
vector of F(oo) and makes the MMPP interval stationary (see [45]). This is 
intuitively clear since the stationary vector of F(oo) means that we obtain 
the MMPP started at an arbitrary arrival epoch. 

In order to find an explicit expression for the cost function, we com- 
pute the Laplace-Stieltjes transform of the matrix F. Since F is 
a matrix, we use matrix operations in order to derive /*(/*), which will 
also be a matrix. The interpretation of the elements f*j(fi) are given by 
]g^ e -M>.+i 1{ Xji+i=j .} \ X n = i\ for n > 1, where 1 is the indicator function. 
Let I denote the identity matrix, then is given by 
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/*(/*) = / e-» n F{&t)= / e-^e^-^iA-Q^A-Q^Adt 
Jo Jo 

/•OO 

= / e-^ I - Q+A)t dtA = {nI-Q+A)- 1 A. 
Jo 

Now we can compute f m {n) = P(5 m > 61+ ■■■ +6 n ) = E[— A*X)a=i ^*]- 
The next lemma shows that this is simply given by the product of f*(fi) 
with itself. Note that we do not need the assumption of independence of the 
interarrival times to derive this result. 

Lemma 52. Let f* (11) be the Lap lace- Stieltjes transform of F , where F is a 
transition probability matrix of a stationary arrival process. Then 



E exp(-ny^6 k ) = q 



fc=i 



/*(/*) 



e. 



Proof. Define a matrix Q n with entries Q n {i,j) given by 



Q„(i,j) = E 



exp 



v fc=i ' 



Note that Qi is given by f*{n). By using the stationarity of the arrival process 
it follows that Q n {i,j) is recursively defined by 

m 

Qn(i,j) = y]Q n -i(i,l) E[ exp(-/i<S„) l{x„=j} = l] 

1=1 

m 

= ^Q„-i(t,0 E[ exp(-yLi<5i)l {Xl=i} |X = /] 
1=1 

= 5]g.-i(i,/)-gi(/,j). 
1=1 

Note that the last line exactly denotes the matrix product, thus Q n = Q n -i ■ 
Q\. By induction it follows that Q n = (Qi) n - Then it follows that 

-A*2>*) =£2>(*o = OQn(M) = g f*(fi) 

fc=i ' t=i i=i 

The last equation holds since the row vector 5 is the initial state of the Markov 
process and summing over all j is the same as right multiplying by e. rj 

Hence g(ir) is given by 



178 



8 Networks with no buffers 



K P(m) 



»M = :EE ~Q+ A)' 1 A] 




m=l j=l 



Note that although in case of two servers we know the structure of the op- 
timal policy, it is not intuitively clear that it is optimal in the case of the 
MMPP. The following argument will clarify this statement. Suppose that one 
has an MMPP with two states. Choose the rates Ai and A2 of the Poisson 
processes such that the policies would have period 2 and 3 respectively if the 
MMPP is not allowed to change state. One could expect that if the transition 
probabilities to go to another state are very small, the optimal policy should 
be a mixture of both policies. But this is not the case. 

Markovian Arrival Process 

The Markovian arrival process model (MAP) is a broad subclass of models for 
arrival processes. It has the special property that every marked point process 
is the weak limit of a sequence of Markovian arrival processes (see Asmussen 
and Koole [18]). In practice this means that very general point processes can 
be approximated by appropriate MAP's. The utility of the MAP follows from 
the fact that it is a versatile, yet tractable, family of models, which captures 
both bursty inputs and regular inputs. 

The MAP can be described as follows. Let {X n \ n > 0} be a continuous 
time irreducible Markov process with k states. Assume that the Markov pro- 
cess is in state i. The sojourn time in this state is exponentially distributed 
with parameter 7*. After this time has elapsed, there are two transition possi- 
bilities. Either the Markov process moves to state j with probability Pij with 
generating an arrival or the process moves to state j ^ i with probability 
without generating an arrival. 

This definition also gives rise to a natural description of the model in 
terms of matrix algebra. Define the matrix C with elements dj = 7* q^ for 
i ^ j. Set the elements Ca equal to — 7*. Define the matrix B with elements 
Bij = jiPij. The interpretation of these matrices is given as follows. The 
elementary probability that there is no arrival in an infinitesimal interval of 
length dt when the Markov process moves from state i to state j is given 
by Cij dt. A similar interpretation holds for B, but in this case it represents 
the elementary probability that an arrival occurs. The infinitesimal generator 
of the Markov process is then given by C + B. Note that a MMPP can be 
derived by choosing C = Q — A and B = A. 

In order to derive an explicit expression for the cost function, we use the 
same approach as in the case of the MMPP. The transition probability matrix 
F(t) of the Markov renewal process {(X n , S„) \ n > 0} is of the form (see [87]) 




8.11 Robot scheduling for web search engines 179 

Again the elements of the matrix F(t) have the interpretation that Fij(t) 
is given by V(X n+ i = j,6 n+ i < t\X n = i) for n > 1. It also follows that 
F(oo) defined by — C~ X B is a stochastic matrix. Let the row vector s be 
the steady state vector of the Markov process. Then s satisfies the equations 
s(C + B) = and se = 1. Define the vector row vector q = sB/sBe. Then 
q is the stationary vector of F(oo). This fact can be easily seen upon noting 
that sB = s(C + B — C) = s(C + B) - sC = -sC. With this observation 
it follows that qF(oo) = (sBe)- 1 sCC^B = q. The MAP defined by q, C 
and B has stationary interarrival times. 

The Laplace- Stieltjes transform /*(/t) of the matrix F is given by 

/*(/*)= / e-" It F(dt)= / e-^e 01 {- C){- C~ l )B At 
Jo Jo 

e -(/*/-o)t dtB = _ c yi Bm 

The interpretation of /* is given by the elements f*j(fi), which represent 
the value of the expectation E[e /1 *"+ 1 l{x„ +1 =j} I X n = i\ . By Lemma 52 we 
know that f m {n) is given by the product of /*. Therefore the cost function, 
when using the MAP as arrival process, is given by 

K P(m) 

" 1-1 
m=l j=l 




8.11 Robot scheduling for web search engines 

In [10] we specified a routing problem where the expected average weighted 
loss rate was to be minimized (or equivalently, the average weighted through- 
put or average weighted number of packets at service was to be maximized) . 
This gave rise (Lemma 47) to an immediate cost of the form: 



c(x, a) = E exp 



i=l 



Due to stationarity of the interarrival times, this cost function satisfies Con- 
dition (8.7) (with a m = C = 0). We assume of course that bi are not all zero, 
which then implies the strict convexity of f m . Indeed, denote 



y = exp 



k=2 J 



Let x be a state such that x a = m > for a particular action a. Since the 
interarrival times are a stationary sequence, 
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c{x,a) =Eye~^ 6m + 1 = Eye~^ aSl , 
c(x + e a ,a) = Eye^ IJ "' [Sm + 1+Sm + 2] = Eye~ IJ -'' [Sl+Sm + 1 \ 
c(x - e a ,a) = Ey. 

Since the function r(x) := ye~ flaX is convex in x, it follows that r(8i +z) — r(z) 
is increasing in z, so that 

r{6i + S m+ i) - r{6 m+ i) > r(&) - r(0). 

By taking expectations, this implies the convexity. Thus the results of the 
previous sections apply. In this section we present another application studied 
in Coffman, Liu and Weber [42] under assumptions that are more restrictive 
than ours. 

The World Wide Web offers search engines, such as Altavista, Lycos, 
Infoseek and Yahoo, that serve as a database that allow to search information 
on the web. These search engines often use robots that periodically traverse 
a part of the web structure in order to keep the database up-to-date. 

We consider a problem where we assume that there is a fixed number of 
K web-pages. The contents of page i is modified at time epochs that follow a 
Poisson process with parameter fa. The time a page is considered up-to-date 
by the search engine is the time since the last visit by the robot until the next 
time instant of modification; at this point the web-page is considered out-of- 
date until the next time it is visited by the robot. The times between updates 
by the robot are given by a sequence 6 n . In Coffman, Liu and Weber [42] these 
times are assumed to be i.i.d., but in our framework we may allow them to 
form a general stationary sequence. 

Let Oi denote the obsolescence rate of page i, i.e. the fraction of time 
that page i is out-of-date. Then the problem is to find an optimal visiting 
schedule such that the sum of the obsolescence rates Oi weighted by specified 
constants d is minimized. A reasonable choice for the weights q would be 
the customer page-access frequency, because the total cost then represents 
the customer total error rate. The case where the customer access frequency 
Ci = kfa is proportional to the page-change rate fa is reasonable under this 
interpretation, since the greater the interest for a particular page is, the more 
likely the frequency of page modification is. 

We now show that this problem is equivalent to the problem studied in 
Section 8.8. Indeed, the robot can be considered as the controller in the 
previous problem. An assignment of the n th packet to server i in the original 
problem corresponds with sending the robot to page i and requires 6 n time 
in order to complete the update. The lengths of the periods that page i is 
up-to-date corresponds to the service times of packets before server i in the 
original problem. Page i is considered to be out-of-date when server i is idle. 

Let Si be an exponential random variable with parameter fa. The cost 
which one incurs when sending a packet to server a should reflect the expected 
obsolescence time. Some straightforward computations yield 
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+ 



c(x,a) = k^ a E 



^ - s a 



i=l 



+ k/i a E 



~i + 



= kfj, a x a E6i + k 



E exp - n a ^ h J - 1 



i=i 



This cost function clearly satisfies Condition (8.7). Hence the theorems from 
the previous sections can indeed be applied to this scheduling problem. 



Remark 15. The assumption that the weights a are proportional to the page- 
change rates in is essential in this problem. The cost function for general q 
is given by 

c(x, a) = c a x a E6i + — 

When the q are not proportional to the page-change rates, then the cost 
function is of the type mentioned in Example 7. Therefore if for some i, jf- 
is sufficiently large (in comparison to others) then it becomes optimal never 
to update page i. This is an undesirable situation and it shows that the 
problem is not always well posed when the costs are not proportional to the 
page-change rates \n. 



E exp - n a 6j\ - 1 



i=l 



Remark 16. This problem of robot scheduling for web search engines illus- 
trates the importance of including the terminal cost Ct in terms of modeling. 
Indeed, for any finite horizon T, if we wish that the cost represents indeed 
the obsolescence time of a page, we have to make sure that if this page is 
never updated (or at least it stops to be updated after some time), this will 
be reflected in the cost. It is easy to see that the terminal cost defined in 
Section 8.8 indeed takes care of that. 
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9.1 Introduction 

We consider in this chapter the control of vacations in several queuing set- 
tings. This chapter is mainly based on [7]. Vacations are time periods during 
which the server does not serve customers, even when there are some in the 
system. 

We consider three types of vacation models: 

(i) vacations driven by service completions, 

(ii) vacations driven (triggered) by arrivals, in which both the beginning as 
well as the end of a vacation are related to arrivals instants, and 

(iii) the potential vacation times form a renewal process and the arrival epochs 
are stationary subsequence of this renewal process. 

We shall consider two types of problems. In both cases, we consider open- 
loop control where the controller has no information on the state of the 
system. 

In the first problem, there is a single infinite FIFO queue. Some vacation 
opportunities are presented (depending on the type of vacations, these op- 
portunities are triggered by service, or arrivals or by some other mechanism). 
The server should go on vacation during a fraction of at least 

p of these opportunities. The goal of the control is to minimize the average 
workload or waiting time (or any nondecreasing convex functions of these). 

The second problem concerns a polling model. There are several infinite 
queues; when serving one queue, the server is unavailable for other queues. 
We wish to minimize some linear combination of the average workloads in the 
different queues (or of waiting times, or of nondecreasing convex functions of 
these). 

9.1.1 Organization of the chapter 

The structure of the Chapter is as follows. In Section 9.2 we formulate the 
two types of control problems: the one of the single queue (PI), and the 
one of optimal polling of several queues (P2). We then formulate the four 
generic type of results obtained for these problems. In the following sections 
we then present and analyze the different models for the vacations and derive 
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the appropriate results for the control. In Section 9.3 we analyze the case 
where vacations are triggered by service completion. In Sections 9.4 and 9.5 
we consider an arrival driven vacations. Finally, in 9.6 and 9.7 we consider 
models where the vacations are a renewal process. 

9.2 The generic control models and main results 

We consider two generic control problems in this paper. We shall formulate 
these problems in an abstract setting, and then focus in the following sections 
on specific assumptions. 

Constrained model: 

— Customers (or some demand for service) arrive to a single G/G/l queue 
(or to a network) according to some given stationary process. 

— There is a single server at some output of the service facility (queue or net- 
work) that may be either active in providing service, or may be absent for 
vacation periods. Some "vacation opportunities" are presented, in which 
that server can decide whether to take a vacation or not. These opportu- 
nities would depend on the model we consider: they could be related to 
arrivals, to end of services, or to be a renewal process. 

— At the nth vacation opportunity, the controller chooses a control a n that 
determines the number num(a n ) of vacations to be taken at the nth vaca- 
tion opportunity. Let a be the control sequence (oi, ci2, ...)• 

— Performance measures and objectives: 

- Let h : K — ► M be a convex nondecreasing function. 

- Given some fraction p, consider the class II (p) of all policies that satisfy 
the constraint: 

1 3 

liminf - num(a n ) > p. (9.1) 

s— too S 4 

71=1 

Consider the following problems. 

(PI) The Vacation control for one queue 

(Pla): Let W n be the waiting time of the nth arriving customer. 

Define the average expectation of the function h of the waiting time: 

1 s 

g(a) =limsup-^Eft(W„(ai,...,a n )), (9.2) 

s — too <S 

71=1 

The objective is to minimize g(a) over a £ II{p). 

(Plb): Let V n be the workload in the system at some special time instants 
T 

Define the average expectation of the function h of the workload: 
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g'(a) = limsup -'y]Wi(V n (a 1 , ...,o„)). 



(9.3) 



The objective is to minimize g'(a) over a £ H (p). 

Next we present our first generic result that will be established in the follow- 
ing sections for different models. To that end we introduce some definitions. 
Let p and 6 be two positive reals. We will use the bracket sequence {a£(#)} 
with rate p and initial phase 9: 



where [x\ is the largest integer smaller than or equal to x. 

In the different models that we study in the next sections, we shall show 
the following for both problems (Pla) and (Plb): 

Result 45 There exists some ratep* such that for any 6, the sequence a\ (6) 
is optimal. 

In Section 9.3, we shall establish Result 45 and show that p* = p, where 
p is given in constraint (9.1). In all other sections where (PI) is considered, 
we shall have p* =1 — p. The difference is simply due to different definitions 
of the control in different models that we study. 

Next we describe the nonconstrained problem (P2): 

(P2) Polling of several queues 

— There are K queues to which a server is allocated. When serving one queue, 
the server is unavailable for other queues. 

— Again, some "vacation opportunities" (or "switching opportunities") are 
presented, in which the server can decide to stop serving one queue and 
start serving another one. 

— At the nth vacation opportunity, the controller chooses a control a n = 
(aj,, ...,(1%); for each n, all components of a n are except one component 
that may be 1 or 0, a z n = 1 will mean that the server is assigned to queue 
i at the nth opportunity instant. Performance measures and objectives: 
Let hi : K — ► ffi be a convex nondecreasing function, i = 1,...,K. 
(P2a): Let be the waiting time of the nth arrival to queue i. Define 



where f^(a l ) = Ehi(W^(a)) and W l n depends on a only through a 1 . 
The objective is to minimize g{a). 

(P2b): Let V* be the workload in the ith queue at some special time 
instants T n . Define 



a' 



?(o) = [k P + e\ - [(k-i)p + e\ 



(9.4) 
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g'(a) d = lim sup ^ ( ^ /;(a l ) 




where /;(a l ) a ^ Eft, 



The objective is to minimize g'{a). 

Next we present generic results for problem (P2). To that end we introduce 
some further definitions. For any vector 9 € R K (which is called a phase 
vector) and a rate vector p e [0, 1] K , we define the vector valued sequence 



Note that a(p,6) need not correspond to a policy since it may have more 
than one component that equals to 1 for the same n. In that case we say that 
it is not feasible, if it defines a policy, we say that it is feasible. 

The following results for problems (P2a) and (P2b) follow from the results 
in Chapter 6. 

Result 46 Assume that K = 2. There exist some p* and 6 such that a(p*,6) 
is a feasible policy and is optimal. 

Result 47 Consider an arbitrary K. Suppose costs and service disciplines 
are symmetric for all queues. Then the round robin policy is optimal for (P2a) 
(resp. for (P2b)). 

And more generally, for K > 2 we have: 

Result 48 The sequence of functions /„ allows one to construct some convex 
function / : M. K -> 1 as in Chapter 1 with the following property. Let p* 
be the vector that minimizes this function. Assume that there is a sequence 
of numbers {i n } n , where i n £ {1,...,K} such that for every k £ 1,...,K the 
sequence aj; = l{i n = k} is bracket with rate p* k (for some 6 that may depend 
on k). Then {a^} are optimal for (P2a) and (P2b). 

The main tool for obtaining the above results is again by establishing the 
multimodularity of some sequence of functions /„ : Z — *■ ffi, where Z is 
some convex subset of Z™, the set of n-dimensional vectors of integers. 

9.3 A single queue with service driven vacations 

Consider a single G/G/l queue (Problem (PI)). The nth customer arrives 
at time T n , bringing a workload of a n to the system. Customers are served 
according to the FIFO order. The arrival process will be assumed to be a 
point process throughout the paper, unless otherwise stated, and we assume 
that T < < Ti < T 2 < .... 



a{p, 6) by 



(9.5) 
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Let r n = T n+ \ — T n denote the inter-arrival times. When a service of a 
customer is completed, the server is allowed to go on vacation. We consider the 
so-called repeated vacation model, where on each completion of a vacation, 
another vacation can be initiated. 

In this model, "vacation opportunities" are thus triggered by the end of 
a service or of a vacation. 

Let a = (ai,02...) be the server's policy, where % £ N has the inter- 
pretation of the number of vacations to be taken after the ith service time 
completion. (In terms of the notation of Section 9.2, we have num(a n ) = a n .) 

Let v n ,n = 1, 2, ... be the duration of the nth vacation period. Let m(n) 
denote the number of vacations that occur till the n + 1st service starts. We 
set m(0) = 0. Denote by 

m(n) 

S n d =a n + Yl v i ( 9 - 6 ) 

j=m(n-l) + l 

the total delay related to the nth customer. It is the sum of its service time, 
plus the vacations that will take place after its service. The waiting time W n 
of the nth customer is given recursively by 

W„+i = (W n -T n + S n ) + . (9.7) 

In particular, assume that the system is initially empty. If no vacations are 
taken before the service of the second customer then m(l) = 0, and the 
waiting time of the second customer is 

W 2 = (Wi - n + 5i)+ = (-n + <ri) + 

If, instead, the 1st vacation is taken just after the service of the 1st customer, 
then m(l) = 1 and 

W 2 = (Wi - Tl +5i)+ = (-Tl +CT1 +v 1 ) + . 

Let V n = V n (a) be the virtual workload in the system immediately after 
the nth arrival; it is defined to be the total time required by the server to 
serve all the customers actually in the system (including the one that arrives 
at time T n ) plus all the vacation times that will elapse from the arrival instant 
T n until customer n + 1 is served. V n is given by 

V n =W n + S n . (9.8) 

def 

Fix some integer N. W(a) = Wn+i{(i) can be written as 

JV 

W(a) = max(0, w\ +x,W2, wjv), where Wj = Wj(a) = y^(5j — Tj). (9.9) 
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Here, x = is the initial workload in the system. Define V(a) = Vjv(a). 

Denote E„ the expectation over the v's (for given random realization of r 
and a). Below we shall establish the multimodularity of K u h(W(a)), where 
h is any nondecreasing convex function. The dynamics of the vacation model 
resemble those of the admission control model in Chapter 4. 

Property 4- The following holds for < i < N. If > 1 then 

Wi(a- Si) = Wi(a) +i> m (i_i), Wj{a- Si) =Wi(a) for j ^ i. 

Note that (— Si) corresponds to adding a vacation after the end of service of 
the ith customer, and delete the last vacation from the i — 1st one. 

Property 5. Consider a vacation sequence v = (vo,...), and the shifted se- 
quence: v' = (v' , v[ , ...) = (vi, v 2 , ...). Let w\ be defined as Wi in (9.9) with 
the sequence v' replacing v. The following holds for < i < N. 

w'jia + ei) = Wi(a) j > i, 

Wj{a + ei) = Wi(a) + u m (j_i) j < i. 

Lemma 53. Assume that v is a stationary sequence. Let h : M — ► ffi be a 
nondecreasing convex function. Then E„ h(W (a)) and K„h(V(a)) are multi- 
modular in a. 

Proof. We consider the basis T = (ei, — s 2 , — sjv-i, — e m ) and check the 
condition h{W{a - v)) + h{W{a - w)) > h{W(a)) + h{W{a -v- w)), v ^ w. 
Case 1: we check for Si,Sj,i ^ j. 

W{a- Si) = max(W{a),Wi +v m ( i _ 1 j), 

W(a- Sj) = max(W{a),Wi +v m ( j _ 1 j), 
W(a- Si - sj) = max(V7(a),Wi +v m ^ 1 j,w j +v m ( j _ 1 j). 
If W(a) is maximizing in the above equation, then 

h(W(a - Si)) + h(W(a - sj)) = 2h(W(a)) = h(W(a)) + h(W(a - s< - sj)) 

and the condition is satisfied. It is then also satisfied for V(a) since ^(o) = 
Wjv-i(a) + Sn{ci), and Sn{<i) is the same for a, a — Si, a — Sj and a — Si — Sj. 
If the maximizer is Wi + iv^-i), then W(a — Si — Sj) = W(a — Si), and the 
condition follows from the monotonicity of h. By symmetry we obtain the 
argument for j instead of i. The same argument holds for V(a). Since this 
inequality holds samplewise, it also holds in expectation. 

Case 2: we check for the first term of the basis. It corresponds to adding an 
additional vacation vq after the service of the first customer. In order to check 
the inequality for the expectation, we consider the following coupling. We 
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consider a second system defined on the same probability space. Quantities 
in the new system will be denoted with an over-line. We let v n+ \ = v n for all 
n. 

We compute W(a) and W(a — Si) in our original system, and compare 
them to W(a + e\) and W(a — Si + e\) in our new system. 

W(a + ei) = max(V7(o), w\ + v ), 

W(a + ei - Si) = max(V7(a), w\ + v , w\ + v + ^ m (i-i)), 

and W(a — Si) = max(W(a),wi +^ m (i-i)). The condition for the multimod- 
ularity holds for both h(W) and h(V) by arguments as in Case 1. Since this 
inequality holds for any sample, it holds in expectation. 

Case 3: we check for the last term of the basis. It corresponds to removing 
the last vacation v m ^y 

W(a-e N ) = (W(a)-v m{N) )+ (9.10) 
W{a - e N - Si) = max{W{a) - v m ( N) ,Wi +v m {i - 1) - u m (iv)) + (9.11) 

If the argmaxof the last maximization is 0, then W(a — ejv) = W(a — ejv — Si) 
and the multimodularity condition is seen to hold (since h is nondecreasing). 
If it is not 0, then W(a — Si) — W(a — ejv — Si) = v m ^y Hence 

W(a) - W(a - ejy) < W(a - Si) - W(a - e N - Si) = v m ( N y 

Since h is convex nondecreasing and W(a — — Si) > W(a — e^), 

h(W{a-Si))-h(W(a-e N -Si)) 
= h{W{a - e N - Si) + i> m (jv)) - h(W{a - e N - Si)) 

> h{W{a - e N ) + v m( jv)) - h{W{a - e N )) 

> h(W(a) - h(W(a-e N )). 

Next, we check this case for the workload. We have 

V(a-e N ) = V(a)-v m(N) (9.12) 
V{a- e N - Si) = V{a- Si) -v m ( N) (9.13) 

Since h is convex nondecreasing, and since V(a — Si) > V(a), this implies 
that 

h(V(a))-h(V(a-e N )) 

= h(V(a) + v m(N) ) - h(V(a)) < h(V(a - Si ) + v m(N) ) - h(V(a - Si )) 
= h(V(a - Si)) - h(V(a - e N - Si)). 

Thus the multimodularity condition holds for h(V) as well. 

Again, the condition for the multimodularity holds samplepathwise, and 
thus in expectation. | 
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Theorem 49. Consider problem (PI), where the expectation E in (9.2) is 
with respect to the random sequences <j,t,v, and where num(a n ) = a n . As- 
sume that 

— the inter-arrival and service time sequence (r„,<7„) is stationary, and is 
independent of the sequence v (t and a may depend on each other), 

— the v sequence is stationary, 

— the following stability condition holds: the queue is in a stationary regime 
at time 0, corresponding to the policy that does not take vacations (see 
more details in Remark 17 below). 

Then Result 45 holds for (PI a) and (Plb), where p* = p is the fraction given 
in the constraint (9.1). 

Proof. The proof is based on Theorem 6. For any integer n, the function 
def 

fn{a) = %-,a,vh(Wn{ai, a n )) is nondecreasing in each en, i = 1, ...,n. 
Moreover, 

fk(ai,...,a k ) = / m (0^0, ai,...,a k ), k<m, (9.14) 

m — k 

(this implies conditions < 2 > and < 3 > in Section 1.3) and it is multimod- 
ular (condition < 1 > in 1.3). 

The monotonicity condition follows from Property 5, the stationarity of 
the vacation times, and the fact that the vacations are independent of the 
interarrival and service times. 

The second condition is satisfied due to the stationarity assumptions. 
Indeed, since the system is assumed to be in a stationary regime at time 0, 
corresponding to the policy that does not take vacations, the Palm probability 
P^r (of the process seen at the times T n ) is invariant under the shift f? Tl 
(see, e.g. [21] p. 19). In particular, if we do not take vacations till time n, 
then Wfc(o) = Wi(a), k < n in distribution. Hence, the distribution of W n 
under the policy a is the same as the distribution of W n +j under the policy 
a' = (0, 0, ai, a k , ...), for any j > 0. This implies (9.14). 

3 

The multimodularity condition was established in Lemma 53. | 

Remark 17. A sufficient condition for the stability condition in Theorem 49 
is that 

— {T n ,<J n ) is a stationary ergodic sequence, and 

— Eo-i < En 

(see [34, 35]). (This sufficient condition also implies coupling to the stationary 
regime from any initial state, provided that we do not take vacations.) 

Remark 18. Throughout, when we say that a sequence is stationary ergodic, 
then we mean with respect to the 1-step shift, unless otherwise stated. Note 
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that this implies the stationarity under any fc-shift but not the ergodicity 
under that shift. Indeed, define a n = (— l) n , ^ n = —a n , »eR Let a be a 
random sequence with P(a = a) = P(a = 7) = 1/2. Then a is ergodic with 
respect to the 1-step shift (see [35]). It is a stationary sequence (with respect 
to the fc-shift for any integer fc, but it is not ergodic for a shift operator •& of 
def 

two steps, (i?a)„ = a n+ 2- In particular, the expectation of ao does not equal 
to the sample average of the sequence am- 

Extension to an arbitrary network 

Let queue i be one of several possible output queues of an arbitrary net- 
work. Assume that every customer that is served in that queue leaves the 
system. Then the waiting time of the nth customer equals to its sojourn time 
till it arrives to that queue, plus its waiting time in queue i. Since customers 
served at queue i are not rerouted, the first component of the waiting time 
does not depend on the polling strategy. The total waiting time of a cus- 
tomer is thus the sum of a part that does not depend on the control, plus 
the waiting time in a G/G/l queue which is influenced by the controller of 
the vacation. It is thus multimodular, due to the results of the first part. 
Hence the optimality of a bracket policy for the total average waiting time is 
directly obtained. 



9.4 An arrival-driven vacation model 

Consider a single G/G/l queue (problem (PI)). The nth customer arrives 
at times T n , bringing a workload of a n to the system. Let r n = T n+ i — 
T n denote the inter-arrival times. Immediately after an arrival occurs, the 
server may go on vacation, that lasts till the next arrival occurs. Then it 
may go back serving, or take another vacation, etc. A vacation policy a = 
(ai, d2, ...) indicates, for each n, whether the server goes on vacation (a n = 0) 
or continues serving (a n = 1) immediately after the nth arrival. (In terms of 
the notation of Section 9.2, we have num(a n ) = 1 — a n .) 

We call this system an arrival driven vacation model since the beginning 
and end of vacations are initiated by arrivals. 

In this section, and in all following sections that deal with problem (PI), 
the constraint (9.1) translates to the following one, directly on the rates of 



The waiting time W n (a) of the nth arriving customer is given recursively 




(9.15) 



by 



W„+i(a) = (W„(a) + a n - a„r„) 



(9.16) 



or explicitly by 
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>V„ + i(o) = max(0,wi, w 2 , w n ), where Wj = ^(o"j - (ijTj). (9.17) 

j=i 

The workload in the system immediately after the nth arrival is given by 
V n = W n + <J n - Note that it satisfies the recursion 

V n+1 (a) = (0, V n (a) - a n T n ) + + a n+1 . (9.18) 

The equation (9.16) seems dual to the dynamics of the admission control 
in Chapter 4. Therefore, it seems natural to expect to obtain the same type of 
multimodularity results, and therefore also the optimization results. In order 
to obtain multimodularity in Chapter 4, it is necessary to let a n , the nth 
service time, be the service of the nth customer actually accepted. Thus, the 
service time of a customer that is rejected is not denned. Then stationarity 
conditions are assumed on this a n sequence, rather than on the sequence of 
service times of all customers (both the ones accepted as well as the ones 
rejected). 

We thus proceed similarly, and define r' n to be the duration of the nth slot 
during which a vacation was not taken. These are the effective interarrival 
times, since, as we see in (9.16), only these have influence on the dynamics 
of W„ and V n . 

Let k(n) < =^'52i=i a '- Then 

W n+ i = max(0, VV„ + a n - a n T k{n) ). (9.19) 

To see why (9.19) holds, we first note that it agrees with (9.16) for those 
n's for which a n = 0. On the other hand, if a n = 1 then r' k ^ = r„, so (9.19) 
again agrees with (9.16). 

If we now assume that r' n (and not r„) is a stationary sequence, we could 
expect to obtain results dual to those of the admission control. 

However, this does not seem natural: it would mean that the vacation 
control decisions influence the actual interarrival times. In the case of i.i.d. 
interarrival times, however, both r' n and r n are stationary. We shall hereafter 
use the i.i.d. assumption. 

Let 

The waiting time of the n + 1st customer is given by W n +i = (W„ + S n ) + . 
The workload just after this arrival is V n = W n + <J n . 

Remark 19. Since a n does not depend on a, V n is multimodular if and only 
if W„ is. 

def 

W(a) = Wjv+i(a) is given explicitly by 

JV 

W(a) = max(0, wi, W2, wjv), where Wj = Sj . (9.20) 

3=i 
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We can now obtain the multimodularity of the expected waiting time as 
we did in the first section. The corresponding properties are: 

Property 6. The following holds for < i < N. If a; > 1 then 

Wi(a + Si) = Wi(a) + T k ^, Wj(a + Si) = Wi(a) for j =fi i. 

Note that Si corresponds to removing a vacation at time Tj_i and adding 
the vacation at Tj. 

Property 7. Consider the sequence of effective interarrival times r' = (tq, ...), 
and the shifted sequence: i?r' = (t[,t' 2 , ■■■)■ Let w'i be defined as Wi in (9.20) 
with the sequence i?r' replacing r. The following holds for < i < N. 



Denote Er the expectation over the effective interarrival times (for a fixed 
realization a). 

Lemma 54. Assume that r n are i.i.d. Let h : ffi — ► ffi be a nondecreasing 
convex function. ThenK T h(W{a)) andK r h(V(a)) are multimodular in a. 

Proof. The proof for the expected waiting time is the same as the one of 
Lemma 53, with r' k ,^ replacing v m ^y The proof for the expected workload 
follows from Remark 19. ■ 

Theorem 50. Assume that 

(i) the service time sequence a is stationary, 

(ii) the interarrival times (r„) are i.i.d. and independent of a, 

(Hi) the queue is initially (at time 0) in a unique stationary regime cor- 
responding to the policy that never takes vacations. 

Then Result 45 holds where p* = 1 — p, and where p is the fraction given 
in the constraint (9.1) (or (9.15)). 

Proof. The proof is based on Theorem 7. We need only check that 
For any integer n, the function 

def 

fn{a) = %- i0 h(Wn{ai, ...,a n )) is non-increasing in en, i = 1, ...,n, (9.21) 
and it satisfies the following conditions: 
— condition < 2 > in Section 1.3: 



w'j(a - ei) = Wi(a) j > i, 
w'j(a - ei) = Wi(a) + t'^^ j 



< i- 



fk(ai,...,a k ) > /fe-i(a 2 , ...,a fe ),Vfc > 1 



(9.22) 
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— a sufficient condition for < 3 > in 1.3: 

/fc(oi, ...,o fc ) = / m (l^^l,ai,...,ajfe), k < m, (9.23) 

m — k 

and 

— fk{o) is multimodular (condition < 1 > there). 

The monotonicity follows directly from the explicit solution to the Lindley 
equations (9.17). (9.22) and (9.23) follow from Property 7, the assumption 
that the initial state is initially in the stationary regime corresponding to no 
vacations, the fact that r n are i.i.d. (and thus stationary), and the fact that 
they do not depend on a n . For (9.22) we also make use of the monotonicity 
property. | 

In Theorem 50, we assume that the queue is initially (at time 0) in a 
unique stationary regime corresponding to the policy that never takes vaca- 
tions. Let Po be the corresponding distribution of the initial waiting time Wi. 
We next show that the results hold for other distributions f3 as well. 

Lemma 55. (Relaxing the assumption on the initial condition) 

Consider any initial distribution f3 ofW\. Assume that conditions (i) and (ii) 

of Theorem 50 hold and instead of condition (Hi), the following is satisfied: 

— ft is stochastically larger than the flo, 

— (a, t) is a stationary ergodic under the k-shift for any integer k, 

— the stability condition Ea < (1 — p)Er holds. 

Then the results of Theorem 50 still hold. 

Proof. For any policy o, both the waiting time as well as the workload at 
any time n are strictly nondecreasing in Wi as can be seen from (9.19). By 
the definition of stochastic ordering, the expectation of any nondecreasing 
function of Wi is larger for the initial distribution j3, than for (3q. This im- 
plies that the average expected costs g(a) and g'(a) are larger for the initial 
distribution (3. In order to establish the theorem it suffices thus to show that 
for the optimal policy, the average expected costs do not depend on the initial 
state. 

If p (in the constraint (9.1)) is rational, then the (candidate for the) 
optimal policy a p k (0) (with p* = 1 — p) is periodic. In that case, the process 
corresponding to the optimal policy starting from any two different initial 
states couple in a time that is finite with probability one. We may couple 
now the initial states, i.e. construct a common probability space where the 
initial state corresponding to [3 is larger than the one corresponding to flo . It 
follows that the convergence of the difference between W n corresponding to 
ft and to flo is monotone decreasing. This implies that the difference between 
fn{o) = Et,ct h(W n {ai, o ra )), starting at the different initial distributions 
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of Wi, converges to 0. Hence the expected average cost under the two initial 
distributions is the same under the optimal policy. 

The same convergence (and hence the same conclusion) is obtained for 
p irrational. Indeed, if p is irrational, then the (candidate for the) optimal 
policy a v (6) is aperiodic. This cost obtained by that policy is unchanged 
if we replace 6 by a random variable 0, uniformly distributed in [0, 1] (this 
follows from the discussion in the end of Section 1.3.1). The policy a v {&) 
is stationary ergodic with respect to the 1-step shift. Then we can use the 
theory of (non-controlled) stochastically recursive sequences by Borovkov [34] 
pp. 260-272 and [35] to obtain the same convergence results as above. | 

Remark 20. The assumption that j3 is stochastically larger than the /3o is not 
really restrictive. Indeed, if j3f is the distribution of the state at time t, then 
one can show that for any policy it, lim inf t ^oo (3™ is stochastically larger 
than the j3o. Hence the assumption is suitable for the case where the system 
has operated for a sufficiently long time under an arbitrary policy. 

9.5 Arrival-driven polling model 

We now analyze problem (P2). Consider K queues, each of which behaves like 
the one in the previous section. The service period for one queue constitutes 
a vacation for the others. 

The nth customer arrives at time T n , and brings K jobs to the K queues: 
a workload of a % n arrives to queue i, i = 1,...,K. These components are 
processed according to the FIFO order in each queue. 

Service beginnings and vacations are synchronized with arrivals. More 
precisely, T n is also the time at which the nth potential service begins; it may 
be in any one of the queues. The service time duration r n is the difference 
between consecutive interarrival times: T n+ \ = T n +T n . If queue i is the nth 
to be served and is empty then we assume that the server still remains r„ 
time at that queue. 

Let be the waiting time of the nth job arriving to queue i, and V* 
denote the workload at queue i just after the arrival of the n customer. The 
evolution of the waiting time VV^(a) in queue i is given by: 

W n+1 {a) = max(0, W n (a) + < - <r„), 

where a z n = 1 if queue i is served at the nth period, and is, otherwise, zero. 

A policy is a sequence a = (ai, 02, ...,), where a n = {a 1 ^, k = 1, K}, as 
defined in Section 9.2. We adopt the further constraint that for every integer 
j, only one of the components aj, i = 1,...,K may be different than 0. Denote 
a 1 = (a\, a>2, ...) the actions corresponding to queue i. 

The following is a consequence of Section 1.4 and the properties estab- 
lished for a single queue in the previous section. 
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Theorem 51. Consider problems (P2a) and (P2b). Assume that 

— the inter-arrival times are i.i.d., and independent of the service times, 

— the service time sequence in each queue is stationary, 

— for each i = 1, K, queue i is initially at a unique stationary regime that 
corresponds to the policy that never takes vacations at that queue. 

Then Results 46, 47 and 48 hold. 

Again, we may relax the assumption on the initial distribution, as we did 
in Lemma 55. 



9.6 The potential vacation times are a renewal process 
9.6.1 A single queue 

Let u n be an increasing random sequence of potential switching times. Imme- 
diately after u n , the server may decide to go on vacation till the next instant 
u n +i- As in the previous section, a vacation policy a = (ai,a>2, ...) indicates, 
for each n, whether the server goes on vacation (a„ = 0) of continues serving 
(a n = 1) at time u n . 

Let s n be the sequence of differences between consecutive potential switch- 
ing times. Thus u n+ \ = u n + s n+ \. 

The fcth customer arrives at time = u nk , where rik is some increasing 
sequence of positive integers. Thus, u n can be viewed as basic time epochs to 
which both arrivals and vacations are related. However, unlike the model in 
the previous section, where arrivals occurred at beginning of each vacation 
slots, arrivals are only synchronized with u n , and need not occur at every pe- 
riod. This will allow us to handle dependent arrival times, and more precisely 
the case of stationary interarrival arrivals. 

Customer fc brings a workload (request for service time) of a* k . Hence, the 
amount a n of workload that arrives at time u n is given by 

<r B = (°. ... *"f»*' V * (9.24) 
[ a k if for some k,n = n^. 

The waiting time W n of the (possibly virtual) customer that arrives at 
time u n in the system can now be computed using the following recursion: 

W n +i(a) = max(0, W„(a) + a n - a n s n ). 

The workload V n at the nth time epoch (i.e. immediately after u n ) is W n (a) + 
a n . It can also be given recursively as 

V n+ i(a) = max(0, V n (a) - a n s n ) +a n+1 . (9.25) 
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We are now back to the model described by (9.16) of section 9.4, and there- 
fore, the multimodularity results in Lemma 54, and the optimality results in 
Theorem 50 hold. 

It is useful to present conditions directly on the original service sequence 
a* (instead of the sequence a which are used in Theorem 50) for the opti- 
mality results. 

In order to make general and yet useful probabilistic assumptions on a n 
(i.e. on the marks of the arrival process), we use the stochastic point process 
formalism. The sequence n k , which we used in (9.24), defines a discrete time 
point process (Af, ■&, P) (where •& is the 6\ of [21] p. 43): 

M{^C) = Y J ^ ) {C). 

kez 

Thus, 

for w £ J7, an arrival occurs at time u n (u>) if Af(co, {n}) = 1. (9.26) 

We associate to the process Af the marks a\ . We assume that Af is com- 
patible with the i? flow, i.e. 

rifc(w) = n (i?jfew). 

Assume that the interarrival times are a strictly stationary sequence, i.e. 

PjU^n* e •) = PJUO. k e ^ ( 9 -27) 

where P^ is the Palm probability related to Af. Then there exists a probabil- 
ity measure P for which Af is stationary (w.r.t. (i?, P)). This follows from the 
inverse construction of Slivnyak (see p. 27 in [21]) in a discrete-time version 
(which follows from p. 44 in [21]). Define 

a{ l ) = (J t for n k <l<n k+1 . 

Then (Af,a) are jointly stationary (with respect to (■&, P)) as follows from 
the argument in [21] p. 13-14. Since 

a n = W(n) x Af({n}), 

it then follows that a n are stationary w.r.t. (i?, P). Indeed, 

<j n {uj) = a(n,co) x Af(w, {n}) = a(n - x Af(<du>, {n - 1}) = o- n _i(i?w). 

(9.28) 

We conclude that if we assume that the original process a* is stationary, then 
there exists a probability measure under which Af is stationary (w.r.t. (■&, P)) 
(thus in particular, the process a n will be stationary). Note that, in general, 
there may exist other non-stationary processes Af that have a stationary Palm 
distribution. 
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If we assume that the service times are independent of the vacation op- 
portunities times s n and of the sequence rik , then a simple argument shows 
that the stationarity of a* implies that a n are stationary too. Indeed, assume 
as above, that <r* are the marks of the process Af, and assume that (9.27) 
holds. Fix some integer j and let Si, S2, Sj be some Borel sets in M. Then 

PK e Si, ...a* e Sj) = p ( a i* G Si,. ..a* e s>i = fc)P(m = fc) 

00 

= J] P(ctJ e Si, ...a* e 5,|m = o)P(m = Jfc) 

k= — oo 

= P(^ e e s^|m =0) 

= P> 1 *6 5 1 ,...<r*e5 3 ). 

Hence, the stationarity of cr* under P^ implies that it is stationary under P, 
and if u n are i.i.d. then the process Af is stationary (w.r.t. (1?, P)). Thus, as 
in (9.28), we conclude that a n are stationary. 
We summarize this in the following Theorem. 

Theorem 52. Assume that 

— the inter-potential vacation times s n are i.i.d. and hence u n is a renewal 
process, 

— arrivals occur at u nk , where rik defines a point process M, 

— the service times <r* are marks of the point process Af, 

— a* is a stationary sequence, 

— the durations of the potential vacations s n do not depend on the service 
durations and on the sequence rik, 

— the queue is initially (at time 0) in a unique stationary regime correspond- 
ing to the policy that never took vacations. 

Then Result 45 holds where p* = 1 — p, and where p is the fraction given in 
the constraint (9.1) (or (9.15)). 

Proof. We show that Theorem 50 can be applied. As we showed above, we 
can consider an equivalent model where arrivals occur at each time u n instead 
of the original ones. The service time for this new model are a n , which are 
stationary. Due to the independence between s n ,nk, and the service duration, 
the interarrival times in the new model are independent of the service times. 
The conditions of Theorem 50 are thus satisfied. (Note that the fact that in 
the new model, arrivals occur at times u n which are independent of other 
quantities, allows to have dependence between the rik sequence.) | 

Note that we may relax the assumption on the initial distribution, as we 
did in Lemma 55. 
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9.6.2 The polling control problem 

Having seen that the setting described in the previous subsection for a single 
queue can be embedded into the one in Section 9.5, we can obtain the cor- 
responding results for the optimal control problem (P2) of polling to several 
queues. 

Theorem 53. Consider problems (P2a) and (P2b). Assume that 

— the potential switching times u n are a renewal process, 

— the service times a*' 1 of the nth customer at queue i,i = 1,...,K are sta- 
tionary, 

— arrivals to queue i occur at times u nic ^,i = 1,...,K, where rik(i) is a 
stationary point process, 

— the duration of the slots s n+ i = u n+ i — u n do not depend on the service 
durations and on the sequences rik{i), 

— for each i = 1, K, queue i is initially at a unique stationary regime that 
corresponds to the policy that never takes vacations at that queue. 

Then Results 46, 47 and 48 hold. 

Note that the fact that service times in different queues were allowed 
to be dependent in Theorem 51 allows us to have dependence between the 
nk(i),i = I, K sequences in different queues. 

We may relax the assumption on the initial distribution, as we did in 
Lemma 55 and Remark 20. 

Remark 21. The assumptions of Theorem 53 contain as a special case the 
following exponential model. Suppose the arrival process are independent 
Poisson process with rates Xj for queue j = 1,...,K. The service time in queue 
j is exponential with parameter fij, i = 1, K. The potential switching times 
form a Poisson process with parameter v > (Ai + ... + Xk), this is a natural 
assumption on v as in case we want to uniformize all processes, v is taken 
as YliLii^i + l 1 ^- Also as approximation of continuous-time polling control 
we may take v large. Theorem 53 now shows that the optimal polling control 
is bracket for the exponential model with K = 2 and for the symmetrical 
model with K > 2. This shows Property 1 in paper [66], where an algorithm 
for computing optimal policies is given. 

9.7 1-gated service 

We describe in this section models that have stationary arrival processes 
which may be more general than point processes. The vacations opportunities 
in the following models will be a periodic process, independent of inter- arrival 
times or service times. 
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9.7.1 A single queue 

We now consider a vacation as in the previous section, but with a "contin- 
uous" arrival into a single queue. We assume that the total workload that 
arrives during the interval (u n ,u n +i] is a n . This workload might arrive in a 
single batch, or continuously, or at several distinct instants in that interval. 
If the server does not go on vacation on time tt„, then the amount of service 
given to the queue till u n +i = u n + s n+ i is the minimum between s n+1 and 
V n (the workload present just before the nth potential switching-interval). 
Thus, only workload present in the gating epoch u n is candidate to be served 
during the interval (u n , u n +i] - We assume that if the server is not on vacation 
at the beginning of the nth slot, then it remains in that queue till time u n +i, 
even if there is no workload to be served during a part (or all) the interval. 

We shall require that the process a n be stationary in n (i.e. w.r.t. the shift 
i?i). The recursion (9.18) for the workload in the system at gating instants 
holds in our case, so we could obtain again optimality of the bracket policy 
(as in Theorem 50). 

In order for the conditions of Theorem 50 to hold we need, however, 
that a n be independent of s„. This is impossible in general, unless s„ are 
identical, which we shall thus assume below. (For example, assume Poisson 
arrivals with rate A, where each customer requires a unit of workload. Then 
the expected amount of workload arriving during a period v„, conditioned 
on s n , is \s n . Hence it is not independent of s„). Note that this problem did 
not occur in previous sections, since the service time was not related to the 
arrival instants, but to the order of arrival. 

Consider an underlying probability space (J7, F). Define { 
vartheta t }, t £ M to be a measurable flow on (f2, T) (see [21] p. 8 and Remark 
22 below). We define n £ Z to be another measurable flow on (f2, T) 
(see [21] p. 44); i?„ will be related to shifts in discrete time. 

Consider a general random measure Z describing the arriving workload; 
in particular, if C is an interval in M, then Z(u>, C) has the interpretation of 
the amount of workload arriving during that interval for a realization oj. This 
includes in particular the case where the arrival process is a point process. 
Assume that Z is stationary with respect to (i?t , P) (i?t is the continuous time 
shift). Then the amount of workload a n that arrives during the deterministic 
(constant) periods s n (= sq) is stationary in n (i.e. w.r.t to (i?i,P)) due to 
the stationarity of Z w.r.t. (i?t, P). Hence the conditions of Theorem 50 hold 
for any stationary arrival process (not necessarily a point process). 

Remark 22. Consider a more general model for the potential vacation pro- 
cess. Let (N, i? t ,P) be a stationary point process corresponding to the po- 
tential vacations: associated with N there is given the random sequence u n , 
n = 1,2, .... We have 




(9.29) 
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where we assume — oo < ... < it_i < uq < < u\ < u-i... < oo and S x is the 
Dirac measure at x. a n can then be considered as marks of the point process 
N: 

Let Z be stationary with respect to (■#(, P). Then 
fnM d = f Z(w, [tt„,tt„ + i)) 

satisfies 

<t„(w) = (To w) = cto^w). 

Hence, ((A 7 ", cr), i? t , P) is a stationary marked point process (see [21] p. 10) 
and a n is stationary in n (see Section 1.3.2 in [21]), i.e. w.r.t. 

To summarize, we have: 

Theorem 54. Assume that 

— the potential vacation durations (s n ) are constant, 

— the sequence of workloads a n arriving arriving during the nth slot is sta- 
tionary, 

— the queue is initially (at time 0) in a unique stationary regime correspond- 
ing to the policy that never took vacations. 

Then Result 45 holds where p* = 1 — p, and where p is the fraction given in 
the constraint (9.1) (or (9.15)). 

We may relax the assumption on the initial distribution, as we did in 
Lemma 55 and Remark 20. 

9.7.2 The case of several queues 

Consider K queues, each of which behaves like the one in the queue in Sub- 
section 9.7.1. The service period for one queue constitutes a vacation for the 
others. 

More precisely, let u n , n= 1, 2, ... be the time at which the nth potential 
service ends; it may be in either one of the queues. The service time duration 
s n is the difference between consecutive inter-switching times: u n+ i = u n + 
s n+ \. If queue i is the nth to be served and is empty then we assume that 
the server still remains s n time at that queue. 

Let a l n be the amount of workload that arrives to queue i during the in- 
terval (u n ,u n+ i]. As in Subsection 9.7.1, we assume a 1-gated regime, where, 
only workload that is present at time u n is candidate to being served during 
the interval (u n ,u n+ i], and not workload that arrives during that interval. 

The evolution of the waiting time W^(o) in queue i is given by: 

W l n+1 (a) = max(0, W*(o) + < - a>„), 
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where a z n = 1 if queue i is served at the nth period, and is, otherwise, zero. 
Here, has the interpretation of the waiting time of a customer that would 
arrive at time u n , and V£ is the workload in the system just after time u n . 

From the discussion in Subsection 9.7.1, we obtain the following results 
(with notation similar to those in Theorem 51): 

Theorem 55. Assume that 

— the arriving workloads is a stationary sequence for each i, 

— s n are constant, 

— for each i = 1, K, queue i is initially at a unique stationary regime that 
corresponds to the policy that never takes vacations at that queue. 

Then Results 46, 47 and 48 hold. 

We may relax the assumption on the initial distribution, as we did in 
Lemma 55 and Remark 20. 

Remark 23. Note that we allow in this model for different (dependent or 
independent) arrival streams (and thus interarrival times) to different queues, 
unlike the model in Section 9.5. The restriction in Section 9.5 to a single 
sequence T n that defines the time of arrivals, that occur simultaneously to 
all queues, was due to the fact that it was the arrival times that triggered 
the polling (the vacation opportunities). In this section, arrival can be more 
general. (Note that, even if the arrival is a point process, the interarrival times 
{T^} in queue i do not appear explicitly anymore in the evolution equations, 
due to the gating.) 

We illustrate the usefulness of the previous result in the following optimal 
scheduling control problem in an telecommunication (ATM) switch. 

9.7.3 Application to an ATM switch 

We consider anMxJV switch with M inputs and N outputs, as depicted in 
Fig. 9.1. We assume that there are separate input queues for each output, so 
we do not have HOL (head-of-line) blocking. Each input is associated with 
N queues, one for each output. We denote by queue ij the queue for cells 
arriving to input i and destined for output j. We consider a slotted queuing 
model where in each time slot at most one cell can be transmitted from 
each of the M inputs, and at most one cell can be received by each of the N 
outputs. In ATM (Asynchronous Transfer Mode), indeed packet size are fixed 
and constant, so it is natural to consider time-slotted models. A scheduling 
mechanism decides at each time slot, from which inputs and to which output 
do we send a packet. 

The scheduler may send simultaneously packets from different inputs to 
different outputs, as long as the following constraints are met: 




CI: The scheduler cannot send more than one packet from the same input 
simultaneously, and 

C2: it cannot send more than one packet to the same output, simultaneously. 

We are interested here in open-loop scheduling policies, i.e. in scheduling 
that do not rely on queue length information, but only on the input rates 
(that will be detailed below). 

A class of policies have been presented in [13], that achieve 100% through- 
put of the switch. A natural problem is whether one can obtain a policy that 
not only achieves the above goal, but also minimizes nondecreasing convex 
functions of the workload in the system. 

Note that a policy that minimizes the workload, maximizes the amount 
of workload that departs, and therefore, the throughput. Therefore, if any 
policy achieves maximum throughput (and stability), then so does the policy 
which minimizes the workload. 

For all 1 < i < M, 1 < j < N, let Aij{n) be the number of cells that 
arrive at queue ij in time slot n. We assume that the arrival process {Aij} n 
is stationary with rate \ij (i.e. the average number of cells arrived in each 
time slot). The arrival processes may be mutually dependent. 

We consider the symmetric case below, i.e. we assume that the Xij do not 
depend on i (they may depend on j), and that N = M. For any i = 1, to, 
define j(i, t) = (i+t)mod(N) + 1. Consider the round- robin scheduling policy 
u that sends at time t a packet (if there is any) from queue i to queue j(i, t) 
for each i. This policy clearly meats the constraints CI and C2 above. For 
each j, all queues ij, i = 1,...,N (having j as destination) receive a round- 
robin service, which, under the conditions of Theorem 55, is optimal among 
all (open-loop) scheduling policies (in fact, even among those that do not 
satisfy CI). 

A scheduling policy is a sequence a = (ai, a-i, ), where a n = {a%,i,j = 
1, ...,N}. If the ijth component of a n is 1, this means that the server polls 
queue ij at the nth time slot. 

Let hij =Hea convex nondecreasing function. Define for each j, 

r n j (a ij )=Eh(W l n j (a ij )),i,j = l,...,N, 
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The following Theorem is then a consequence of Theorem 55: 

Theorem 56. Consider the above N x N switch. Assume that 

def 

— the arriving workloads a z J = Aij{n) is a stationary sequence for each i. 

For each j, the distribution of the processes of arrivals of workload to the 

queues ij does not depend on i (in other words, f%{a l i) = /^(a*-*) if 
a ij = a kj j ; 

— the time slots s n are constant, 

— the workload initially in each queue corresponds to a unique stationary 
regime that would be obtained if this queue had always been served before. 

Then the round-robin policy u minimizes both g(a) as well as gi{a), for j = 
1,2,...,JV. 

Again, we may relax the assumption on the initial distribution, as we did 
in Lemma 55 and Remark 20. 
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10.1 Introduction 

This chapter deals with the problem of closed loop control (unlike the pre- 
vious ones, which focused on open loop control) for stochastic discrete event 
systems which are not necessarily Markovian. We present a new approach, 
based on multimodularity, to show monotonicity properties of the optimal 
policy which works under various types of the information patterns: total 
information, sampled information, delayed information. 

The concept of multimodularity can be found in applications in feedback 
control [52, 80, 114, 27] where multimodularity properties of the value func- 
tions were formulated in terms of the number of customers in two parallel 
queues, or in tandem queues. We call this notion multimodularity in "space". 
It must be distinguished from multimodularity in time, that we are using and 
that will allow us to obtain monotone properties of the optimal policies which 
are state dependent. 

Moreover, the types of networks for which the multimodularity in space 
is known (basically Markovian systems) is much more restricted than those 
for which multimodularity in time holds, as illustrated in this chapter, by 
some examples. It is thus quite desirable to have a theory that would re- 
late multimodularity in time with the structure of state-dependent optimal 
policies. As far as the proof techniques are concerned, most of the works 
on space monotonicity uses a dynamic programming approach and expected 
costs (see for example [52, 27]). Here, we will primarily use a sample path ap- 
proach. Some steps in this direction have already been taken in [15], in which 
some sample-path properties that are related to multimodularity are used to 
establish the monotonicity of optimal policies in the state (or some partial 
information that we may have on the state). In this chapter we pursue this 
goal and construct a general theory for the state-dependent optimal control 
of discrete event systems with time-multimodular cost. 

We consider several types of questions related to different information 
structures: (i) monotonicity in the initial state, (ii) monotonicity in the cur- 
rent state in the case of full state information, (hi) delayed and sampled 
information, and others. We show for all these problems that some simple 
multimodularity properties of the costs imply the monotonicity of optimal 
policies. 
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Other general frameworks have been studied in the past for establishing 
monotonicity of optimal policies, such as [12, 52, 80]. The conditions are re- 
lated to submodularity, and were typically formulated for Markovian models. 
In particular, [52] consider only exponential distributions. Our methodology 
allows for non-Markovian analysis, as multimodularity of workload and wait- 
ing times was established for general stationary ergodic sequences of service 
and interarrival times (see Chapter 4). 

10.1.1 Organization of the chapter 

The rest of the chapter is organized as follows. In Section 10.2, we define the 
multimodular ordering on action sequences. In Section 10.3, we introduce the 
different information patterns that we will consider. In Section 10.4, we inves- 
tigate the special case of full information whereas the general case is treated 
in Section 10.5. Finally Section 10.6 shows that the traditional state repre- 
sentation also fits in this framework. The relations between Multimodularity, 
superconvexity and submodularity is shown in Section 10.7. 

10.2 Monotonicity in initial actions 

— Problem P: Consider a function /„ : A n — > M, and assume that /„ is 
multimodular (A™ is the action space and will be considered as a bounded 
convex subspace of W 1 ). Assume that ai,...,a„_i are fixed, and we wish 
to minimize /„ with respect to a n . 

We wish to establish properties of optimal (In ctS ct function of a±, a^-i- 
In other words, we wish to characterize the impact of some initial choice of 
actions, which need not be a result of optimization consideration, on the 
future choice of actions. 

In order to establish monotonicity properties, we introduce some ordering 
on the set of action vectors. 

Definition 15 (m-order). Let a 1 , a 2 £ A n . We say that a 1 < m a 2 if there 
exists a sequence 71,. ..,7; wherein £ {+ei, — S2, — S3, — s n }, such that a 2 = 
a 1 +7i +72 + ••• 

Theorem 57 Consider problem P. Then there exists optimal control a„(a) 
which is monotone nonincreasing in the initial controls a = (ai, ...,a„_i). 

Proof. Let a 1 < m a 2 where a i € A™ -1 . For any 6 i e A, i = 1, 2 with 6 1 > b 2 
we have 

U{a 2 ,b 1 )-f n (a 2 ,b 2 )>f n (a\b 1 )-f n (a\b 2 ). (10.1) 
Indeed, if we write x = (a 1 , b 2 ) then the inequality can be rewritten as 



f n (x + w- me n ) - f n {x +w)> f n {x - me n ) - f n (x) 



(10.2) 
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where m is some positive integer, and where w is the sum of some elements 
in T . This inequality then follows from the definition of multimodularity. 

Now, assume that b 1 is optimal at a 2 but not at a 1 . We shall show that 
this implies that b 2 is not optimal either at a 1 . This will then imply that 
there exists an optimal policy which is monotone nonincreasing. 

Indeed, due to the optimality of b l at a 2 we have 

f n (a\b l )<f n (a 2 ,b 2 ). 

(10.1) then implies that 

Ua 1 ^ 1 ) < f n {a\b 2 ). 

Since b 1 is not optimal at a 1 , then b 2 , which does not perform better, is not 
optimal either. We conclude that an optimal control at o 1 has to be larger 
than b 1 . 

Thus if we define the policy that chooses for a given sequence a % the 
maximal element b that is optimal at a\ then this policy is monotone nonin- 
creasing. This ends the proof. □ 

We say that a set B C A is decreasing if b £ B implies that b' £ B for 
any b' < b. To illustrate the above Theorem, assume that A = {0, 1, ... , M}. 
Then the theorem implies that there exists an optimal control a n (-) such that 
for each < m < M, there exists some decreasing set B m c A™ -1 and such 
that a e B m if and only if o„(a) > m. B m is thus the set of initial sequences 
of length n — 1 for which the corresponding optimal action at instant n is 
greater than or equal to to. 

Definition 16. The maximal elements of B m defined in the previous para- 
graph are said to be the rath switching curve that characterizes the optimal 
action a n (-). 

We may extend Theorem 57 to the two dimensional case (this extension 
will be useful in Model 60 in the next section). 

Corollary 58 Consider an action space A = {0, 1}. Consider two functions 
f l n : A n — ► R, i = 1,2, and assume that f % n are multimodular. For any given 
sequence a n = (ai, a n ) € A", define b n = (1 — ai, 1 — a n ). Assume that 
ai,...,a„_i are fixed, and we wish to minimize the function 

g n (a n ) = fiiaT) + f 2 (b n ) 

with respect to a n . 

Then there exist optimal control a n (a) which is monotone nonicreasing in the 
initial controls a = (ai, ...,a„_i). 

Proof The proof follows directly from Theorem 57 by noting that f 2 , and 
thus g n are multimodular with respect to a 71 . □ 
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10.3 Exogenous random variables and information 
patterns 

We consider in this section the cost functions that depend not only on the 
control but also on some random exogenous variables {Z n }. More precisely, 
let {Z n } be a sequence of random variables taking values in some space Z, 
and let there be a sequence of cost functions: /„ : A™ xZ"->l 

We assume that /„ are multimodular in the control actions a for each 

realization of z n = {zi, . . . , z n }. 

A history of length n has the form (ai, . . . , a„_i; Z\, ... ,z n ), and H ra is 
defined to be the set of possible histories of length n. We define the observa- 
tion function y n : H n — ► Y„, where Y n is the observation space available at 
time n. 

A policy u = (tti, it2, . . .) is a sequence of maps where u n is a function 
from Y„ to A. In other words, at time n an action is chosen as a function of 
the available observation of the history. 

We shall consider several information patterns: 

1. Full information: y n (hn) = h n , so that the whole history is available. This 
can be seen as the general full information case. (For the standard MDP 
case, the whole past history can be considered as a state of the system, 
see Section 10.6.) 

2. Sampled information: let k be some fixed integer; we assume that the 
information on the past actions is always available, but the new informa- 
tion about the values of the variables Z n becomes known only at sampled 
times k, 2k, 3k, ... In other words, let 

«») = *l£J- 

Then 

y n {ai, • • • , a„-i;zi, . . . , z„) = (ai, . . . , a„_i; z\, . .. , 

Since no new observation on the z sequence arrives during the period 
l(n) + 1, . . . , l(n) + k, the control under this information structure is 
equivalent to one that takes decisions only at times rk,r = 1,2, where 
at each time rk the actions a r k+i, ■ ■ ■ , ffl( r +i)A are determined. We shall 
adopt this view below. 

3. Delayed information: fix some integer 9. Then the information at time n 
over the z- variables is delayed by 6 units, so that 

y n {a\, . . . , a n -\;z\, . . . , z n ) = (oi, . . . , a„-i;zi, . . . , z n -g). 
Our goal under the different information structures is to minimize 
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JV 

J N {u) = J2^fn{H n ,A n ) + W*g N+1 {H N+1 ) (10.3) 

71=1 

where N is some given integer and gN+i is some terminal cost. This cor- 
responds to a finite horizon problem where N sequential actions are to be 
chosen. 

In order to illustrate our framework we present three models which will 
be considered all along the paper. 

Model 59 Consider a single queue operating at discrete time. An arrival of 
a n packets is generated at the beginning of the nth time unit, where a n is a 
control variable. At the end of the time unit, a departure occurs if and only 
if z n+ i = 1 (otherwise, z n+ i = 0). The number x n of packets in the queue at 
the end of the nth slot (just after a potential departure), which we call the 
state of the queue, is given recursively by 

x n +i = (x n +a n - z n+ i) + . (10.4) 

We assume that at n = the queue is empty. Using the explicit solution of 
this Lindley's equation, be expressed as a function of the history: 

n-l 

x n = max (o, y^(a k - z k+1 ),l= 1, ...,n - l). (10.5) 

k=l 

We may now choose f n (h n ) ^= r n (x n ) where r n is any monotone increasing 
convex function; this will ensure that f n is indeed multimodular in the a- 
sequence for any realization of the ^-sequence, see [6] and Section 4.3 of [4]. 



Model 60 Consider a routing problem into two queues: qO, ql. An arrival is 
generated at the beginning of each time slot. The routing variables a n , n G N 
take values or 1; the nth packet is routed to queue b n = 1 — a n . At the 
end of the time unit, a departure from queue i occurs if and only if z l n+1 = 1 
(otherwise, z l n+l = 0). The number x % n of packets in queue i at the end of the 
nth slot (just after a potential departure) is given recursively by 

X n+1 = ( X n + a n ~ Z n+1 ) + J 
X n+1 = ( X n + bn ~ Z n+ i) + . 

Consider the cost function 

g n ( x n ) = c r n (x° n ) + ciri(xi), 

where d > are some given real numbers and r % n are convex increasing 
functions. It follows from Model 59 that x° is multimodular in a 71-1 = 
(ai, a n -i) and x\ is multimodular in b 71 ^ 1 = (&i, (where b 71 ^ 1 = 
(1 — ai, . . . , 1 — a n -i)). Recalling Corollary 58, we conclude that gn(x n ) is 
multimodular in a 71 ^ 1 . 
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Model 61 We consider here (max,+) linear systems. Any system in this 
class can be modeled by event graphs, which are a subset of the Petri nets, 
as described in Chapter 3. 

As in previous chapters, and unsing the notation from Chapter 3 and 
Chapter 4, let Xi(n) denote the time when transition i initiates its n-th 
firing, then X(n) = (Xi(n),--- ,Xg(n))* satisfies a linear equation in the 
(max,+) notation of the form: 

X{n) = A{n) ® X{n - 1) © B{n) ® T v{n) . 

Define W% = X q (n) — T u ( n y It is the traveling time fox customer n between 
its entrance in the system and its passage in transition q. Again, we have the 
Lindley's equation: 

W n = A(n) ® D(-T n ) ® W n _i © B n , 

where D(h) is the diagonal matrix with h on the diagonal and — oo everywhere 
else. The equation is developed into: 

n 

W n+l =B n ®Q)C„ (10.6) 

i=l 

with 

n 

Ci = <g){A{j) ® D(- Tj )) ®B{n-i- 1), (10.7) 

j=i 

and .B(O) = (— oo, — oo). (see Chapter 3.) 

Assume now that a n is a stationary sequence, independent of the sequence 
r n . Using (10.7), it was shown in Chapter 4 that for any convex increasing 
function r and for any realization of the sequence r„, E aT{W^) (the ex- 
pectation is taken with respect to the distribution of the a n process) are 
multimodular in o. 

In order to be in our general framework, we can identify the sequences a n 
and T n as the Z n variables in the beginning of the section. 

Finally, in order to obtain multimodularity of r(W£) for any realization 
of Z n (and not just the expectation of r(W%)) we shall have to require that 
the <j n == (j do not depend on n (note that they may still be random). The 
randomness thus exists only in the arrival. We note that such Petri nets with 
deterministic firing times and stochastic exogenous arrivals have been the 
subject of several research work, see e.g. [78]. 

Many other models give rise to recursive equations of a form similar to 
(10.4), and hence the same type of results that will hold for Models 59 and 
60 will also hold for these: 
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The G/D/l queue: We consider general independent arrivals and a determin- 
istic (constant) service time requirement of one unit time length. Let T n be 
the arrival epochs, and let z n+ \ be the amount of potential service between 
T n and T n+1 . Hence z n+ i = T n+ i —T n . At the nth epoch T n , the number of 
customers that are accepted is determined by the control and is given by a n . 
Then (10.4) holds where x n denotes the workload at time T n . 

The D/G/l queue: Assume that the time between arrivals is a constant. The 
waiting time of the nth customer that is admitted at the queue satisfies a 
recursion similar to (10.4) (this was established in Theorem 6.4 in [4]). 

10.4 Monotonicity: full information case 

We consider here the the full information case. We consider the cost over a 
horizon of length N. 

Some of the arguments below rely on dynamic programming, which moti- 
vates us to define the cost to go: having observed the actions a r and z r during 
r periods, we define recursively the cost to go 4> r+1 and the optimal cost to 

def 

go g r+ \ as follows. For r < N we set a T = (a r +i), and define 

4> r (a r ;H r ) = E a " (f T {H T ,a T )\H T } +E°" (g T+1 {H T+1 )\H T } 

g r {H r ) = mm 4> r {a r ;H r ). (10.8) 

Let U be the set of policies that choose at time r actions achieving the 
min in (10.8). 

Theorem 62 Consider the control problem under full information. Assume 
that the cost functions f n and the terminal cost gN+i o,re multimodular for 
any realization of the z sequence. 

Then there exists an optimal policy u £ U such that for each integer r < 
N and each history h r , the action a r , is monotone nonincreasing in a = 
(ai, a r -i). 

Proof. By standard arguments of dynamic programming we know that any 
policy in U is optimal. The proof will follow directly from Theorem 57 if we 
show that 4> T [a T , is multimodular for all integers r < N. 
The multimodularity is established as follows. 

— Since /„ and gjv+i are multimodular, it follows that the summation 

/«(-ffw,o«) + 9n+i{Hn+i) 

is also multimodular. 

— Taking conditional expectation, we obtain the multimodularity of cf> N+1 . 
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— Using the Key Lemma in the appendix, this implies the multimodularity 
of g N . 

— Proceeding the same way by induction, we obtain the multimodularity of 
g r for all r < N. 

This establishes the proof. □ 



10.5 Monotonicity: general case and delayed information 

We consider in this section a general information structure. We assume that 
y n (hn) contains the realization of (oi, . . . , a n ) (i.e. we have full information 
on prevoius actions). We further assume that it is monotone increasing in n, 
i.e. y n is contained in y n +i- Other than that it can be arbitrary. 1 

We note that this general information structure includes both the sam- 
pled as well as the delayed information structures. It includes also the noisy- 
delayed information structure studied in Altman and Koole [11]. 

In particular, to see that it includes the case of sampled information, we 
define 

yrk+j(h Tk +j) = (a rk+j - 1 ,z rk ) 

where h n = (a 71 ^ 1 ,z n ), r £ N and j = 1, k. Note that with this interpre- 
tation, decisions can be made at each time unit, but no new information on 
the z's is obtained between information samples. 

We use arguments similar to the case of full information. Define (by back- 
ward recursion) the cost to go and the optimal cost to go as 

4> n (a n ;Y n ) =E a " (/„(#„, a n ) + g n+1 (H n+1 )\Y^ , 

g n {Y n ) = mm (f> n {a n ;Y n ), (10.9) 

iar n = N,N 

Let U be the set of policies that choose at time n actions achieving the 
min in (10.9). 

Theorem 63 Consider the control problem under the above information 
structure. Assume that the cost functions f n and the terminal cost gN+i are 
multimodular for any realization of the z sequence. 

Then there exists an optimal policy u £ U such that for each integer 
r < N and each history h r , u chooses an action a r which is monotone in 
a = (ai, ...,a T -i). 

1 The increasing assumption is necessary for the use of dynamic programming. 
Indeed, dynamic programming is used to identify an optimal Markov policy in 
a MDP setting, where the state is the observation. However, if the information 
is not increasing then given the current observation state y n , the future need 
not be independent of the past. Thus the MDP structure is not satisfied and the 
dynamic programming need not generate optimal Markovian policies. 
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Let's go back to our models 59, 60 and 61. In all of them we saw that 
/„ and g n were indeed multimodular for any realization of the z sequence. 
We conclude that the monotonicity result of Theorem 63 applies to all these 
models. 



10.6 State representation 

The monotonicity results of the first section are in terms of a sequence of 
actions. However, quite often 

— (I) the cost fn{H n , a n ) at time n can be expressed as a function of a n and 
of some simpler quantity x n (H n ), called state, which is a function of the 
history H n . 

— (II) the distribution of the state x n +i(H n+ i) at time n + 1 depends only 
on the state x n {H n ) at time n and on a n , namely: 

P(x n +i(H n+1 ) e •\H n ) = P(x n+1 (H n+1 ) e •\x n (H n ),A n ). 

In other words, we may also say that x n is a random function of x n -i, a„_i 
and z n (z n represents the randomness). 

The advantage in dealing with states is that this allows typically to decrease 
the dimensionality. For example, the state space is often finite and does not 
grow with n, whereas the set of histories grows exponentially with n and need 
not be finite. When such a state representation exists then different histories, 
possibly with different lengths, may be mapped to the same state. 
Model 59 (continued): The same state x may correspond to different his- 
tories; in particular, the history 

en = Zi + i = 0, i = 1, k - 2, Zk = 0, afc_i = x 

implies that X — X 5 where k is an arbitrary integer. 

Next we present simple conditions under which monotonicity in the ac- 
tions, as described in Section 10.5, imply monotonicity in the state. 

Case 1, an easy case: Assume that 

1. the action space is a convex subset of the integers, 

2. As in Model 59, we assume that for any history h n = {a n , z n ), there exists 
some action a = a(h n ) such that x n (h n ) = x n ((a, 0)). Consequently, each 
state can be identified with an action a, and it corresponds to all histories 
for which 

x n (h n ) = x n ((a,0)). (10.10) 

The above set of assumptions now implies a natural ordering over the states. 
Hence, whenever we have monotonicity of the policy in the sequence a in the 
representation of Section 10.5, this implies monotonicity in the state. 
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10.6.1 General full information case 

Now, consider the standard information structure in MDP, i.e. y n (h n ) = 
(xi,ai, . . .,x n ). 

Theorem 64 Consider the control problem under the above information 
structure. Let f : X x A — > M and define J m : Z m x f — ► M such that 

fm{z m ,a m ) = f(x m (z m ,a m ^ 1 ),a m ). Define similarly for a terminal cost 

g N+1 : X - M: sfw+i(^ +1 , a ff ) = 2jv + i(xjv + i(* JV +\ o w )) : Z^ 1 xN" -» 
K. Assume that f and g are multimodular for any z and that the following 
assumptions hold: 

— <1>: x m : H m — ► X m; w/iere X m is some measurable space endowed with 
some partial order, and A C N; 

— <2>: x m (z,a) is monotone in a for any z, i.e. < m implies that 
x m (z,a^) < x m (z,a^). (Equivalently, x m (z,a + Si) < x m (z,a),i = 
2, . . . , m, and x m (z, a — ei) < x m (z, a).) 

— <3>: There exists a(x,z) = (a±, o>n)(x, z) where a m is a function ofx m 
and of z m = (z±, z m -\), m = 1, N with the following properties. 

(i) For any x m and z m , if we assume that x m = x m (a m ^ 1 (x, z), z m ), 
x m = x m (a m ^ 1 (x, z), z m ) and x m < x m thena m (x,z) < m a m (x,z). 

(ii) For any x m < x m there exists some z m = (zi, z m ) such that x m = 

Xm(® (x : Z), Z ), X m — X m ((l {x : z), Z ). 

Then there exists an optimal policy u € U such that for each integer r < N 
and each history h r , u chooses an action a r which is monotone nonincreasing 
in the current x r . 

Proof. Our goal is to minimize (10.3). 

Consider instead the problem of minimizing (10.3) with /„ and and gjv+i 
replacing /„ and gjv+i, where we assume standard information: at time n the 
controller has the knowledge of all previous actions and realizations of z. 

Let A m (a rn ^ 1 , z m ) be the set of optimal actions at time m given the 
previous actions and z m , i.e. the set of actions that minimize the sum of 
immediate cost at time m plus the optimal cost to go as in eq. (10.8). Hence 
we can apply Theorem 62 and choose a policy a = (ai,...,ajv) which is 
monotone in the history where a m € A m . 

Now lets go back to the original problem. Fix some x m and x m such that 
x m < x m - Let z m and a be as in <3>. Define 

QmiXm) — ^m{x^ 

Then the monotonicity of a in its argument implies the monotonicity of a m 
in x. □ 

We shall now illustrate the validity of the assumptions used in the Theo- 
rem in our Examples. 
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Lemma 56. Consider Model 59. If a 71 ^ 1 < m a 71 ^ 1 then for each z n 

x n (a n - 1 ,z n ) < x n (a n - 1 ,z n ). 
Hence condition <2> holds. 

Proof. ^From the associative property of the partial ordering < m it follows 
that it is sufficient to show for all n, a 71 ^ 1 and z n : 

x n (a n -\z n ) < x n (a n - 1 + e u z n ) (10.11) 

x n {a n -\z n ) < x n {a n - 1 - Si,z n ), t = 2,3, n. (10.12) 

Indeed, it is easy to see that both inequalities follow directly from the repre- 
sentation (10.5). rj 

Next, we show that Condition <2> holds for Model 60. To that end, we 
introduce an ordering on X = I 2 . We say that x m > x m if > x Q m and 
% m 5: % m i where x m = (x m , x m ), x m = (x m ,x m ) € X. 

Lemma 57. Consider Model 60. If a 71 ^ 1 < m a 71 ^ 1 then for each z 71 

x n {a n - 1 ,z n ) < x n {a n - 1 ,z n ). 

Hence condition <2> holds. 

Proof. Since for a single queue (Model 59) assumption <2> holds, it clearly 
holds for the component in Model 60, i.e. 

a (1) < m a (2) implies that x° m (z,a {1) ) < x° m (z,a {2) ). (10.13) 

If we expressed the length of queue 1 in terms of the decision variables b m 
i.e. xl n (z,a 7n ) = x m (z,b 7n (a 7n )), then this would also hold for queue 1, i.e. 
b^ < m b^ implies that x^'z, b^) < x^'z, b^). However, since a* = 1 — bi, 
<™ o (2) implies that b^ > m b^ and hence x^z,^ 1 ^) > x^z,^ 2 ^). 
Combining this with (10.13), we conclude that <2> indeed holds. rj 

Lemma 58. Consider Model 59. If for any n, 

%n — Xjiijl , Z ), Xji — Xjiijl , Z ), 

and x n < x n , then there exists a 71 ^ 1 such that 

a 71 - 1 < m a"- 1 

and 

Hence condition <3> holds. 

Proof. The proof holds by a rather straightforward induction on n. rj 
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10.6.2 Monotonicity of the switching curves 

Next we illustrate further structural properties of optimal policies. As in 
Lemma 59, let there be a function x : H m — ► N, let / : X x A — ► M be given 
and define /' as in Lemma 59. Consider the problem: 

min f(x, a). 

a 

We know from the end of Section 10.2 that one may choose the argmin to 
be of a switching curve structure. In our case it means that there exists 
some minimizer u (as a function of x) that behaves as follows. There exist 
thresholds l m , m £ A such that u(x) < to if and only if x > l m . Note that l m 
is nonincreasing, i.e. 

Assume that / is superconvex as in property P2, or equivalently 
f(x, a + 2)- f(x, a + 1) > f(x + 1, a + 1) - f(x + 1, a). 
Then the minimizer u can further be chosen such that the following holds: 

Indeed, assume that at state x + 1 it is optimal to use some action to but 
it is not optimal to use any action a > to at that state. This means that 
f(x + 1, to + 1) > f(x + 1, to), that l m = x + 1. 

Then the superconvexity implies that f{x,m + 2) > f(x,m + 1), which 
means that it is not optimal to use to + 2 at x; thus the smallest state for 
which there exists some optimal action a among a < to + 1 is x. Hence 

^m+l ^ ^ — 1* 

10.6.3 State representation with delayed information structure 

We have established in Section 10.5 monotone properties of optimal policies 
as a function of the previous taken actions for general information structure. 
In Section 10.6, we considered the monotonicity of optimal policies in case 
that a state representation exists. The two aspects can now be combined: 
state representation with more general information patterns. 

In case of general information structure, we cannot expect anymore to 
have monotonicity in the current state, since the current state might not be 
available to the controller. In order to obtain monotonicity in some quantities 
other than the previous actions, new objects with the role of a state should 
be defined. 

To illustrate that, consider the case of delayed information on the state. 
We thus keep the definition of state x in the beginning of this section (re- 
quirements (I) and (II) there); we assume that the information available to 
the controller can be expressed as 
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y n (h n ) = (ai, . . . ,a n -i,xi, . . . ,x n -e). 

We now define an object that will serve as state, and define a new cost 
function: 

£n(hn) = {Xn-0,an-0, ■ ■ ■ , Cln-l) , 

fc n ,a n )=-Ef((X n ,a n )\Z n ). 

It is easily seen that this new state and cost are legitimate in the terms of 
the requirements (I), (II) given in the beginning of the section. In particular, 
if we note the transition probabilities P xa c for the original state of moving 
from state x to a subset C C X given that action a is chosen, then for 
C = (C, a n -e, ...,q„_i), the transition probabilities for the new state are 
given by 

Assuming that there exists a partial ordering on x, one can now define the 
partial order on the new state space as the component-wise ordering: that for 
x for the first component and the m-order between the second action-vector 
component. 

Now, assume that conditions < 1 >-< 3 > of Theorem 64 hold for the 
original state and that / and g defined there are multimodular. Now, let us 
consider the delayed case. We define 

i Oj\ i • • • i Urn — 1 

By definition, we have, 

Mm) 

= E (f(x m (h m ),a m )\x m -g,a m -g,- ■ ■ , o m _i). 

According to the assumption (II) made on x m (h m ) (in Section 10.6), we 
know that it is in fact a function of only x m -i,a m -i and z m . By induc- 
tion, we may say that x m (h m ) is a function of x m -o, a m _i, • • • ,a m -e and 
Zm-e+i,- • • , z m- Therefore, we can write 

E (f(x m (h )\x 

= E (f(x m (h m ),a m )\h m -e,a>m-e, ■ ■ ■ , a m -i) 
= E (f'(h m ,a m )\h m -e,a m -e,- ■ ■ ,a m -i) 
= ^ z m+ i_,,...,z m / (zi, ...,z m ,ai, a m ), 

and hence /' is also multimodular (we take expectation with respect to the 
unknown random variables z m +i-e, —,z m +i)- In a similar way, we can show 
that the terminal cost is also multimodular. 

It remains to check that conditions < 1 >-< 3 > of Theorem 64 hold for 
the new state £„. Clearly < 1 >-< 2 > hold. 
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Next we show that < 3 > holds for the new state. Choose some £ m = 
(%m-6,<Xm-e,--;<Xm-i)- With the policy a defined as in < 3 > for the original 
state, define the new a as follows: 

(ai,...a m -i)(£ m ,z) = ((ai,...,a m -6-i)(x m -8,z m ~ e ~ 1 ),a m -e,—,a m -i) 

With this choice, it is seen that indeed < 3 > holds for the new state. Indeed, 
let £ m < | m with | m = (x m -e,am-8, ...,d m _i). Note that 

) <m (d m _ e ,...,a m _i). 

Assumption < 3 > used for the original model implies immediately that 
(ai,...,a m _ fl _i) < m (di,...,d m _ fl _i). We conclude that a(a: m ,,z) < m a(x m ,z) 
(component-wise), so that < 3 > holds also for the new state. 
We can thus extend Theorem 64 to the delayed case: 

Theorem 65 Assume that the conditions of Theorem 64 hold (for the prob- 
lem with standard information). Then the conditions and the statement of 
that theorem also hold for the case of delayed information. 

10.7 Relation between multimodularity, superconvexity 
and submodularity 

Lemma 59. Assume that the action space is a convex subset of the integers 
and that <1> and <2> from Theorem 64 hold. 
Let f : X x A — ► M be given and define 

) = f(x m (zi,..., +i,a m +i) 

Then f is multimodular for any z if and only if f satisfies: 

- (PI) it is submodular, i.e., for a± < a.i, a; 6 A, i = 1,2, a € A™, 

/(^m(2,o),2; m+ i,ai) — f(x m (z,a),z m+ i,a2) is nonincreasing in a, 

- (P2) it is superconvex (this property is defined e.g. in [80]), i.e. for a = 
(ai, . . . , a m ), 

f{x m {z,a),z m +i,a + 2) + f(x m {z,a + e m ),z m+ i,a) 
> f(x m (z,a + e m ),z m+1 ,a + l) + f(x m (z,a),z m+1 ,a + 1); 

- (P3) for fixed a m+1 and z, f{x m +i{ai, a m , zi, z m+ i),a m+ i) : A™ -> 
N is multimodular. 
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Proof. Assume P1,P2,P3. We have to check that for any z and any v,w e 

T, v ^ w, 

f'(z,a + v) + f(z,a + w) > f(z,a + v + w) + f(z,a), (10.14) 

where T = {-ei,s 2 , ■ ■ ■ ,s m +i,e m+ i}. If v,w $ {e m+ i,s m+ i} then (10.14) 

holds by property P3. 

Let v = e m+ i, w ^ s m+1 . Then x m+ i(z,a + w) < m x m+ i(z,a). Setting 

def def 
oti = a m+ i and a 2 = a m+ i + 1, we see that the submodularity of / implies 

(10.14): 

f'(z,a + w) — f'(z,a + w + v) = 
f(x m+ i(z,a + w),cti) - f{x m+1 {z,a + w),a 2 ) 
< f(x m +i{z,a),ai) - f(xm+i(z,a),a 2 ) 
= f(z,a)-f(z,a + v). 

_ def 

It now remains to check the case where v = e m +i, w = s m +i- Let a = 
{a!,.. . ,a m }; then 

f'(z,a + v) + f'(z,a + w) = 

f(x m +i(z,a),a m +i + 1) + f(x m +i(z,a + e m ),a m +i - 1) 
f'(z,a + v + w) = f{x m +i{z,a + em),a m +i); 

(10.14) now follows from the superconvexity of /, which establishes the mul- 
timodularity of /'. 

To establish the converse, assume now that /' is multimodular. Then 
properties PI and P2 follow directly from Lemma 2.2 (a) in [6], and property 
P3 follows from (a) and (b.ii) in that Lemma. rj 

Now, we give some comments on the generality of assumptions (P1)-(P2)- 
(P3). 

Remark 24- Note that superconvexity is not needed if we restrict to sequences 
whose elements take only the values and 1 instead of taking values in N. 
More precisely, if / is defined on Nx {0, 1} — ► M then we do not have to check 
the case v = e m +i, w = s m +i, since clearly a + v oi a + w are not within the 
set Nx {0,1}. 

Remark 25. Since x m is assumed to be increasing in a, a sufficient condition 
for property (PI) is that f(x m ,oti) — f{x m ,OL2) is nonincreasing in x m , for 
Oil < ol 2 . 

Remark 26. Note that the superconvexity and the submodularity imply that 
/ is integer convex in its second argument. Indeed, 
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f(x m (z,a + e m ),a + 1) - f(x m (z,a + e m ),a) 

< f{x m {z,a),a + 2) - f(x m {z,a),a + 1) 

< f(x m {z,a + e m ),a + 2) - f(x m {z,a + e m ),a + 1) 

where the first inequality follows from the superconvexity and the second 
from the submodularity of /. 

Remark 27. The following superconvexity property, which is similar to (P2), 
is also implied by the multimodularity of a function /' : Z m x A m+1 — ► M: 

f'{z,a + 2e m ,a) + f'{z,a,a + 1) > f'{z,a + e m , a + 1) + f'{z,a + e m ,a). 

Again, this follows directly from Lemma 2.2 in [6]. 

Simple sufficient conditions can be presented for the assumptions of 
Lemma 59, which are satisfied by models 59 and 60 respectively. 

Indeed, let in <1>, X = M be the set of real numbers endowed with 
the standard ordering between numbers. Then (P3) holds if the following 
conditions hold: 

-fix ) is convex increasing in x m for any fixed value of a m +i, 

- x m satisfies the following: 

X m +1 [z,a m )V X m +1 {Z,d m - V - W) < X m +1 (z , d™ - v) V X m +1 (z , d™ - V)) , 

for any z, where v ^ w, and v,w € J 7 , see [6]. Note that Model 59 satisfies 
the above conditions, (see [4]) as well as assumption <2> in Lemma 59. 

Next consider X = M 2 , with the ordering introduced just before Lemma 
57. Assume that 

i) x^ l+1 (2;,a' n ) are multimodular in a m and satisfy for all v / w,v, w e 

xL+i (z, a m ) V < +1 (z, a m -v-w)< x^ (z, a m - v) V x^ (z, a m - w) 

(10.15) 

ii) / is given by 

f(x m+1 , a m+1 ) = r° {x a m+1 , a m+1 ) + r 1 (x^ +1 , a m+1 ) , 

where r z are convex increasing functions. Then assumption (P3) holds, since 
(10.15) implies that r l (x l m+1 ,a m +i) are multimodular in a™. 

10.8 Appendix: Key Lemma 

Lemma 60. Let f : A — > M be a convex function where A is a convex set in 
W 1 . If the function g, defined by 

def 

g(xi,x 2 ,. ■ -Xk-h) = inf f{xi,x 2 ,---x k ), (10.16) 

Xk-h + l,--- 

is finite for all x\ , x 2 , . . . Xk-h, then it is also convex. 
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Proof. We decompose a vector x in A into a vector it and a vector v of size ft. 
Therefore, we can rewrite Equation 10.16 as g(u) = inf„ /(it, v). Let it 1 and it 2 
be any two vectors of dimension fc — ft. Since g(v}) and g(u 2 ) are both finite, 
for all e > 0, there exits v 1 (resp. v 2 ) such that (it 1 ,^ 1 ) £ A and /(it 1 ,!) 1 ) < 
inf^ f{v},v) + e (resp. (it 2 ,i; 2 ) £ .4 and f(u 2 ,v 2 ) = inf„ f(u 2 ,v) + e). 
By definition of g and by convexity of /, we have 

giXu 1 + (1 - \)u 2 ) < /(AfrV ) + (1 - \){u 2 ,v 2 )) 
<Xf(u 1 ,v 1 ) + (l-X)f(u 2 ,v 2 ), 
< A inf f(u\v) + (1 - A) inf /(u 2 ,t>) + 2e. 

V V 

This is true for all e > 0. Therefore, (/(Ait 1 +(1-A)it 2 ) < Ac/(it 1 ) + (l-A)c/(it 2 ). 
□ 

Lemma 61 (Key Lemma). Let / : N fc — ► ffi 6e a multimodular function. 
Then g : N fc "' 1 -> M <Ze/med 6j/ 

de/ 

g(ai,a 2 ,...ak-h) = inf /(ai, a 2 , ■ ■ ■ a k ) 
is afeo multimodular. 

Proof. First, note that it is enough to prove the result for ft = 1. The proof 
for ft > 1 is done by a straightforward induction. 

Let (ei,e2,--- , e^) be the canonical base of M^_. We denote by VJ the 
vectorial space generated by (ei, ■ ■ ■ , e^). We define the function f :R k — > M 
such that / is the linear interpolation of / on the atoms of ffi fc . Theorem 2.1 
in [6] tells us that / is convex. 

Now, let us define ~g : -> M by: 

def — 

§(oi,a 2 ,...o fc _i) = inf /(oi,o 2 ,---a fc ). 
at 

By using Theorem 2.1 in [6], we just have to prove that g is convex and 
linear on all the atoms of . 

For convenience, we introduce the following notation: for x in M.^_ and S a 

set in , P(x, S) is the projection along e k of x on S. Formally, P{x, S) = 
x + de k where d is the real number with smallest absolute value such that 
x + dek £ S. 

Choose m £ N. We denote by E m the set Vk-i © [0,m]e fc . Note that E m 
is a convex union of atoms in M* . 

By continuity of /, for all it £ there exists v m (u) < m such that 

def — — 

9m(u) = inf„< m f(u,v) = f(u,v m (u)). By using Lemma 60, then the func- 
tion g m is convex. For any y, z £ £? m , 
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g m (Xy + (1 - \)z) < Xg m (y) + (1 - X)g m {z). (10.17) 



2 t _ x def 



In a given atom a of Vfc_i , pick any two points it 1 and it 2 . Let x 
Aw 1 + (1 — \)u 2 . The point (x,i; m (x)) belongs to an atom, say A of 
included in E m . 

We consider F* (resp. the upper (resp. lower) hyperface of dimension 
M^_ _1 of A along e„. We project the point (x,v m (x)) on F* and F*. 

p d = P((x,v m (x)),F. t ), 
q d ^ P((x,V m (x)),F*). 

By linearity of / over the segment \p, q], f(x,v m (x)) is a linear combi- 
nation of f(p) and f{q). Moreover, the points p and q belong to E m . By 
definition of v m (x), we also have f(x,v m (x)) < f(p) and f(x,v m (x)) < f{q). 
Therefore, we have f(x,v m (x)) = f(p) or f(x,v m (x)) = f(q). In the following 
we will consider the case f(x,v m (x)) = f(p). The case f(x,v m (x)) = f(q) is 
similar by using the face F* instead of F» . 

We project the points (w 1 ,i; m (M 1 )), (u 2 ,v m (u 2 )) on F». 

P 1 ^ Pdu^Vmiu 1 ))^,), 

p 2d ^ P((u 2 ,v m (u 2 )),F*). 

By linearity of / over the segment \p x ,p 2 ], and definition of v m {x), 

7((x,v m (x))) =J(p), (10.18) 

= A/(p 1 ) + (l-A)/(p 2 ), (10.19) 

> \fiiu\vmiu 1 )) + (1 - X)f((u 2 ,v m (u 2 )), (10.20) 

> /(A^ 1 ,^^ 1 )) + (1 - X)(u 2 ,v m (u 2 ))), (10.21) 

> 7((x,t; m (x))). (10.22) 

Where Equation 10.19 comes from linearity of / over A, Equation 10.20 comes 
from the definition of v m (.), Equation 10.21 comes from convexity of / and 
Equation 10.22 comes from the theorem of Thales. Therefore, 

g m (\u 1 +{l-\)u 2 ) = g m {x), (10.23) 
= J((x,v m (x))), (10.24) 
= \fiiu\vmiu 1 )) + (1 - X)f((u 2 ,v m (u 2 ))(10.25) 
= \gm(u 1 ) + (l-X)g m (u 2 ), (10.26) 

where Equation 10.25 comes from Equation 10.21, which is an equality. 

All these arguments are illustrated in Figure 10.1, which is done with 
dimension k = 2. 
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Fig. 10.1. Illustration of the proof of the key lemma. 



To finish the proof, it is enough to notice that g(u) = linim^oo g m (u), to 
take the limit in Equation 10.17 to establish the convexity of g and to take 
the limit in Equation 10.26 to establish the linearity on atom a. 

□ 
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Part II shows that in general it is very difficult (and sometimes impossi- 
ble) to construct the optimal policy for routing customers in several parallel 
systems. 

One way to overcome this problem is to replace this ambitious objective 
by an easier one: improve on one initial policy. To reach this objective, we 
propose several order relations between policies which all have the following 
underlying idea. Policy a is smaller than policy b if a is more regular than b. 

All the orders introduced in the following three chapters have some rela- 
tions with each other, but they are indeed different. The utilization of one 
of them rather than the other depends on the cost function as well as the 
overall context. 



11 Comparison of queues with discrete-time 
arrival processes 



11.1 Introduction 

In this chapter, traveling times in a FIFO-stochastic event graph are com- 
pared in increasing convex ordering for different arrival processes. As a special 
case a stochastic lower bound is obtained for the sojourn time in a tandem 
network of FIFO-queues with a Markov arrival process. A counterexample 
shows that the extended Ross conjecture is not true for discrete-time arrival 
processes. This chapter is an adaptation of [61]. 

We consider an open stochastic queueing network with one input node. 
The network dynamics are supposed to satisfy a linear recursion in the 
(max,plus)-algebra on R L , as described in Chapter 3. It is shown in Chapter 
3 that the epoch of the beginning of the n-th firing time of a FIFO-stochastic 
event graph (FSEG) satisfies such a linear recursion for each transition (see 
also [25]). 

We recall that a special case is a stochastic network of L single-server- 
FIFO-queues in tandem, with infinite buffer capacity in the first queue and 
finite buffers with manufacturing blocking or infinite buffers in the other 
queues. Note that the sojourn time in this tandem network of L single-server- 
FIFO-queues, is the traveling time to server L plus its service time at L. 
Hence, the comparison results below hold also for the sojourn time. 

Let T n , n = 1, 2, . . . be a stationary sequence of potential arrival epochs. 
The number of arrivals at T n will be denoted by A n . In general A n , n = 
1,2,... may be a stochastic sequence, and A n = I means that £ customers 
arrive at T n . Note that A n = implies that T n is not an actual arrival epoch. 

Two arrival processes are compared with respect to their implied per- 
formance of the stochastic network. We assume that both have the same 
potential arrival epochs, but different ^-sequences, say A\ and A\. Let us 
call these the admission sequences. Let be the firing-time (service-time) 
of the n-th token (customer) in transition (server) j. We assume that 

S n = {S^, . . . , 

is a stationary sequence of stochastic vectors. Note that no independence 
assumption is made on the firing-times, stationarity is sufficient. But, we as- 
sume that for i = 1 and for i = 2 every couple of sequences from {A z n ,T n , S n } 
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are stochastically independent. Let bW% denote the traveling time of the n- 
th arrival to transition q, i.e. the time between its entrance in the stochastic 
network and the beginning of its firing time at transition q. Let a W% be the 
same time of a potential arrival at T n . Recall that at T n there may be no 
arrival, and the arrival time of the n-th customer is in general not T n . With 
we denote the n-th arrival epoch for arrival process i,i = 1,2, i.e. 

k 

Z* =min{fc:^Aj >n}. 

i=i 

Then the arrival time of the n-th customer is, T z ^ ■ We also need the following 
notation, 

Bi = 2£-.Z£_i, * = 1,2, rc = l,2,..., 

where we take Zq = 0. Note that B l n is the n-th inter-arrival length, in gen- 
eral this is not equal to the inter-arrival time. Of course, B z n is a function of 
A\,A\,-.. ,A* zi ; we suppress this in our notation. It is shown in Chapter 4 
for a (max,p/w,s)-linear system that for any transition q, and n = 1,2,..., 
E (,Wj? is a multimodular function of (Bi, . . . , B n ), and 
E aWk i s a multimodular function of (Ai, . ..,A„), where the expectation is 
with respect to T n and S„, n £ N. These multimodularity properties induce 
the convexity results which we use to prove our comparison results. The ar- 
rival processes in the chapter will be generated by a Markov Arrival Processes 
(MAP), for which we assume a Markov process on E, a finite state space with 
intensities X xy , x,y £ E, and an arrival occurs with probability r xy when a 
transition from state x to state y happens. 

In [63] it is explained that a Markov arrival process (MAP) is more general 
than the Markov-modulated Poisson process (MMPP) or the Phase- type re- 
newal process. In [18] it is shown that any arrival process can be approximated 
arbitrarily closely by a MAP. Let us mention the stochastic orders we use in 
this chapter. Random vectors X 1 = (Xl,. . . ,X„) and X 2 = (X 2 , . . . ,X 2 ) are 
ordered with respect to the convex ordering (X 1 < cx X 2 ) (resp. increasing 
convex ordering (X 1 <j cx X 2 )) if 

EhiX 1 ) <Eh(X 2 ) 

for all convex (resp. increasing convex) functions 

h : ffi™ -f M. 

Increasing and decreasing are in this chapter always in the non strict sense. 

In section 11.2 we give a counterexample which shows that the extension 
of the Ross-conjecture is not true in our comparison of queues with different 
admission sequences. In section 11.3 a first comparison lemma is derived for 
admission (inter-arrival) sequences which are comparable in the convex or- 
dering. It is shown, that the potential (actual) traveling times are ordered in 
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the increasing convex ordering. Similar comparison results hold for the sta- 
tionary traveling times. As applications of the lemma, we derive the following 
results: 

1. independent sources have a better performance (in increasing convex or- 
dering sense) than coupled sources, 

2. fixed batch sizes are better than random batch sizes, 

3. fluid scaling improves the performance. 

In a second comparison lemma, derived in section 11.4, actual (potential) 
traveling times are ordered in the icx-ordering for non-integer admission 
(inter-arrival) sequences. Here a " regularization" procedure is given, which 
has been used in the theory on balanced sequences and optimal routing pre- 
sented in Chapters 2 and 6. In section 11.5 we construct the most regular 
arrival process for a fixed arrival intensity, and we call it the Regular Ar- 
rival Process (RAP). We show that the RAP provides a stochastic lower 
bound for any MAP-source with the same arrival intensity. This result (The- 
orem 68) can be seen as the 'Ross-conjecture-theorem' in the comparison of 
discrete-time arrival processes. In the literature on optimal routing to parallel 
queues, it was claimed (cf. [43]) that good approximations could be obtained 
through replacing the MAP-arrival process by a renewal process with approx- 
imately the same arrival intensity. Theorem 69 and Corollary 9 provide the 
proof of these claims. Indeed, the performance of a RAP has stochastic lower 
bounds for arrival processes which are (approximately) renewal processes. 
For a rational arrival intensity, there is a RAP which is renewal with Erlang 
distributed inter-arrival times. By a continuity argument we obtain that the 
renewal-arrival process with constant inter-arrival process gives, for any real 
stationary arrival probability, a stochastic lower bound on the performance. 

11.2 On the Ross conjecture in discrete-time 

In his inspiring paper [98] Sheldon Ross conjectured that the mean waiting 
time in a -/G/l/oo queue with non-stationary Poisson arrival process is larger 
than or equal to the mean waiting time of the M /G/l/oo queue with the same 
arrival intensity. . This paper of Ross initiated a long sequence of research 
papers on this and related problems. Rolski proved the Ross conjecture in 
[97]. Recent publications on this and related topics are [17, 19, 30]. Suppose a 
■/G/l/oo queue with potential arrival epochs given by sequence T n but with 
different admission sequences A 1 and A 2 . Since the potential arrival times are 
fixed and form a sequence of discrete epochs, we prefer to call this a discrete- 
time model, although the T n may have continuous distributions. Suppose A 1 
is time-stationary and A 2 not, but they have the same intensity. Is the mean 
waiting time for the A 1 sequence smaller than or equal to that of A 2 1 The 
^-sequence can be seen as a random environment and one may expect that 
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an extended Ross conjecture holds (cf. [21], where the service process has a 
random environment) which says: 

"The G/G/ l/oo queue in a random environment should be bounded below 
by the corresponding queue where the environment process is 'frozen' to its 
mean values." 

For the setting of this chapter, it is not true as the following counter 
example shows. 

Let T = {T n } be a Poisson process with rate 1. We consider the -/M/l/oo 
queue, and we assume that the S n are i.i.d. with exponential distribution with 
mean 1. Let A\ be distributed as i.i.d. Bernoulli random variables with mean 
1/2. Let A 2 n be distributed as independent Bernoulli random variables, let 
the mean of A\ be p n . We assume that p n is random with mean 1/2. Then 
A\ is the arrival process for which the random environment of A\ is frozen 
to its mean. So the extended Ross conjecture would claim that for W % the 
stationary waiting time for sequence A 1 , i = 1,2 

E W 1 < E W 2 . (11.1) 

Suppose that the sequence {pi,P2,P3, ■ ■ ■) is with probability 1/2 equal to 
| + e, | — e, | + e, . . . and with probability 1/2 equal to | — e, \ + e, | — e, . . ., 
where < e < |. Then E p n = | which is the probability in the ^-sequence. 
It is easily seen that for e = 1/2 we have that W 1 is the stationary waiting 
time of the M/M/l/oo queue with traffic intensity p = |, and that W 2 is 
the stationary waiting time of the GI/M/l/oo queue with inter-arrival times 
which have an Erlang distribution with 2 phases of exponential length with 
mean 1. It is well-known that 

E W 1 > E W 2 , 

which contradicts relation 11.1. In Section 11.5 we will derive a discrete-time 
analogue of the Ross-conjecture-theorem. The above example explains why 
we use regular sequences there. 

11.3 A comparison lemma and its applications 

In this section we derive a first comparison lemma which is a rather direct 
consequence of the multimodularity of the traveling times as function of the 
admission sequence. As we will see, the comparison lemma has some nice 
implications. 

We recall that b W^ ( a W^), i = 1, 2 is the traveling time of the n-th arrival 
(potential arrival at T n ) to a fixed but arbitrarily chosen transition q in the 
FSEG. 

Lemma 62. The following implications hold for any n = 1, 2, . . . 

1) {A\ ■■■Ai) < cx {A 2 --- A 2 n ) => { a W± ■■■ a Wl) < icx ( a W 2 ■ ■ ■ a W 2 ). 

2) {B\ ■ ■ ■ Bi) < cx (B 2 ■■■!?£)=► { b W± ■■■ b Wl) < icx ( b W 2 ■ ■ ■ b W 2 ). 
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Proof. The proof is given for 1); the proof of 2) goes similarly. Note that 
(A{ , . . . , A l n ) £ f>P , i = 1, 2, let h : M — > M be an increasing convex function. 
Then, for i = 1,2, 

W k (Ai,...,A i n )^Eh( a Wi(Ai,...,A i n )) 

is a function from to IL Theorem 18 (for the proof of part 2, use Theorem 
19) shows that W k {A\, . . . ,A\) is multimodular in {A\, . . . ,A\) as function 
on N fc for k = 1,2, . . .. Since W k (A\,. . . ,A\) is independent of A' n for n > k, 
it trivially follows that Wk{A\ , . . . , A l n ) is also multimodular in (A\ , . . . , A l n ) 
if n > k. It then follows from Theorem 1 that is integer convex on Pf 1 
i = 1, 2, k = 1, . . . ,n. 

The rest of the proof is more or less standard. Since (A\, . . . ,A\) < cx 
(j4i, . . . , j4^) we may by Strassen's representation theorem assume without 
loss of generality that 

E{{Al...,Al)\{A\,...,A 1 n )) = {A\,...,A 1 n ). 

From Jensen's inequality we then have, 

W k {A{,..., A\) = W k (E ((Al. ..,A 2 n ) | (A\,..., A^))) (11.2) 
<E(W k (A 2 1 ,...,Al)\(Al...,A 1 n )). (11.3) 

Hence, 

EW k (A 1 1 ,...,A 1 n )<EW k (Al...,A 2 n ). 
Let h : W 1 -> 1 be an increasing convex function. Then 

h(W 1 (A\,...,Ai l ),...,W n (A\,...,Ai l )) 

is an increasing convex function of (A\ , . . . , A z n ) i = 1,2. The first inequality 
below is now a consequence of (11.3) and the increasingness of h, the second 
inequality follows from Jensen's inequality 

h(W 1 (A\---A 1 n )...W n (A\...A 1 n )) 

< h(E {WMl ...Al)\{A\... Al)) . . . E (W n (Aj . . . A\) \ (A\ . . . A\)) 

< E (h(W 1 {A\ ...Al)... W n {Al ...Al))\{A\... Al)). 

Hence, 

Eh(W 1 (A\---A 1 n )...W n (A\...A 1 n )) 
< E h(W\(A\ . . .A 2 n ) . . . W n (A\ . . -A 2 n )). 

□ 
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Let us assume now the following, 

Assumption 66 A % n is a stationary sequence inn £ Z for i = 1,2. 

The following sequences with i = a or b, j = 1 or 2 are well-known as the 
Loynes-sequences (cf.[21]), 

Ml = i W n {AL n ,A i _ n+1 ,...,A i _ 1 ). 

They are monotone increasing in n. Consequently, they have a limit as n tends 
to infinity, which is possibly oo. These limits are called Loynes-variables, we 
denote them as 

iWL^ lim t W 3 n . 

n— >oo 

It is well-known (cf.[21, 34]) that under strong coupling or renovation it holds 
that 

iWL = lim iW>, 

n— >oo 

i.e. it is the time-forward limit. As an immediate consequence of the compar- 
ison Lemma 62 we find, 

Corollary 7. // assumption 66 holds then 

1) (A\,...,Al) < cx {A\,...,AD for all n > 1 => a Wl < a W^. 2) 
{B\ , . . . , B\) < cx (Bl . . . , Bl) for alln>\ => h Wl < b W^. 

Proof. We prove part 1), the proof of part 2) goes similarly. From assumption 
66 we have for all n, 

(A_ n , A_ n+1 , . . . , A_i) = (A 1 , j4 2 , . . . , A n ) 
^cx (^l?) . . . , ^4^) 
dist ( .2 A 2 A 2 \ 

where means equality in distribution. 
From the comparison Lemma 62, 

w 1 < w 2 

a vr n — icx a rv n - 

The monotone convergence theorem then gives, 

W 1 — lim W 1 < lim W 2 — W 2 

a'' oo — 11111 a'' n ilex 11111 a'' n — a""oo' 

n— »oo n— »oo 

□ 

It is well-known that in case of stability of the stochastic networks the Loynes- 
variables are a.s. finite and represent the stationary versions of the traveling 
times. So in case of stability also the stationary versions are icx-ordered. Also 
a multidimensional marginal distribution of the stationary processes can be 
shown (also as a consequence of comparison lemma 62) to be ordered in the 
icx-ordering. 
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11.3.1 Application 1: two i.i.d. MAP-sources perform better than 
2 completely coupled Map-sources 

Consider two MAP-arrival processes, say MAP\i = 1,2, which are indepen- 
dent and have the same distribution. Denote by T l the transition epochs of 
MAP\i = 1, 2, and let T = T 1 UT 2 be the superposition of T 1 and T 2 . Define 
A\ = 1 if T n e T is an arrival epoch of MAP 1 or MAP 2 . Then T = {T n } 
are the potential arrival epochs, and the admission sequence A\ generates all 
arrivals of the two independent MAP's. Consider now two completely cou- 
pled MAP-sources, which is equivalent to one MAP-source which generates 2 
arrivals at any of its arrival epochs. Say this MAP-source is with probability 
1/2 the MAP-source and with probability 1/2 the MAP 2 -source. Define for 
* = 1,2, 

E i _ f 1 if T n is arrival epoch of MAP 1 
n \ otherwise. 

Then 

A 1 n =E 1 n +El, 

and for A\ the admission sequence of the coupled MAP-sources we have for 
1 < to < n that, A 2 m given A\, . . . , A\ is with probability 1/2 equal to 2E] n 
and with probability 1/2 equal to 2E 2 n . Hence, 

E [A-m I ^1 , . . • , A\ ] = A^ 

and therefore, 

{A\,...,A 1 n ) < cx (Al...,A 2 n ) for all n. 

Clearly, the events E z n in MAP 1 are independent of the transition epochs 
T^, i = 1,2. This implies that {A z n } and {T n } are independent for i = 
1,2. Hence we can apply comparison lemma 62 and corollary 7, and find 
that the potential (stationary) traveling times for the i.i.d. MAP-sources are 
in icx-ordering smaller than those of 2 completely coupled MAP-sources. It 
is possible to extend this result to: k i.i.d. MAP-sources which generate £ 
customers at each of their arrival epochs perform better in icx-order than 
I < k i.i.d. MAP-sources which generate k customers at each of their arrival 
epochs. 

11.3.2 Application 2: a fixed batch size is better than random 
batch sizes 

Consider a MAP-source, and assume that at each of its arrival-epochs a 
random batch number of customers arrive, say N n at arrival epoch T n . 
We assume that {N n } is independent of {T n }, and also that {N n } is sta- 
tionary and E Ni = £. Take A\ = I and A\ = N n , n £ N, then 
A\ is the admission sequence with fixed (or frozen) batch size. Clearly, 
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{A\, . . . ,A l n ) < cx {A\ , . . . , j4 2 ), n e N and part 1) of comparison Lemma 
62 and Corollary 7 applies. 

If I = 1 then (B\ , . . . , B 1 ) = (1, . . . , 1) and in order to show that 

{B\ , . . . , B\) < cx (Bl,...,Bl) 
it suffices to verify that for k = 1, . . . , n, 

EB 2 k = l. 
Since A\ is stationary, it follows that 

E A\ -E B\ = 1, 

and E fit = E Bi = 1. Hence, in this case also, the actual traveling times are 
smaller in icx-order for the fixed batch sizes. 

11.3.3 Application 3: Fluid scaling improves the performance 

Consider the following transformations of the time variable t and the state 
variable x, 

t — ► Nt 

x 

Fluid limits are obtained by taking limits for iV — ► oo. Here we take a 
fixed N e N. If we have a MAP 1 with finite state space E and transition 
rates \ xy , x,y € E, and if we divide the time- variable by N, then we get a 
MAP 2 with transition rates j/\ X y After uniformizing both processes such 
that the transition times in both processes are a Poisson process with the 
same parameter A, say MAP 1 (A) and MAP 2 ( A) , we have that a real transition 
in MAP 1 (A) (i.e. a transition in MAP 1 ) is with probability a real transition 
in MAP 2 (A) (i.e. a transition of MAP 2 ). Clearly, we can couple the MAP X (A) 
and MAP 2 (A) such that if T n are the arrival epochs of MAP 1 (A) then the 
potential arrival epochs of MAP 2 (A) are {T n }, and the admission sequence 
is, 

. 2 _ J 1 with probability ^ 
n ~ \0 else, 

where the A\ are i.i.d. and independent of T n . If we take A\ = \ then A\ is 
the admission sequence for MAP 1 (A). 
As in application 2 we have 

(j4j , . . . , A n ) < cx (j4j , . . . , A n , ), 
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where A k = NA\, k > 1. 

The scaling of the state can be done by considering the original service re- 
quirements as a number of packets (possibly of random size) , and taking N 

2 

arrivals instead of one arrival. This gives the A n as admission sequence. So, 
the process corresponding to the A\ can be seen as a fluid scaling of the A n - 
induced process. Mathematically, it is the same comparison as in application 
2. The comparison lemma 62 and corollary 7 imply that, the performance 
of the fluid scaled process is better than that of the original process in icx- 
ordering. 



11.4 A second comparison lemma 

In comparison lemma 62 we had admission sequences = 1,2, where 

A % n gives the number of arrivals at T n . This means that A % n is an integer. In 
the comparison lemma 63, below, the admission sequences are {p z n }, where 
p z n may be any nonnegative real number. For p z = (p\,p2,-- •) with p z n > 0, 
n = 1,2,... we define an integer admission sequence {A z n (p z )} by, 



n n—L 



3=1 



3=1 



where is a random variable, uniformly distributed on [0,1), and where [^J 
denotes the largest integer smaller than or equal to x e . Note that A z n (p z ) 
is random and integer valued, it gives the number of arrivals at T n . For the 
inter-arrival-lengths we proceed similarly given q z = (q\,qli, ■ ■ .) with q z n > 0, 
n = 1, 2, . . ., we define 



3=1 



n-l 



W) = [£<£+*J-[£ff5+« 



3=1 



where 6 is uniformly distributed on [0,1). Note that if p\,p\, ■ ■ ■ ,p z n are all 
integer valued then A z k (p z ) = p z k for 1 < k < n, and any 6 £ [0, 1). Similarly, 
as in comparison lemma 62, we consider the potential traveling times 

■C= aW n (A\(p z ),...,Al(p z )), i = l,2 
and the actual traveling times 



b W z n = b W n (Bl(q z ),...,B z n (q z )), 



1,2. 



Lemma 63. The following implications hold for any n = 1,2, 



1) (pi,- 

2) ill 



,pk) <ox (p 



(? 2 



1 1 • 



{aWl, 



W 1 ) < { Wr 



• • ,6 



{ b w?,..., b wt). 
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Proof. We prove part 1), the proof of part 2) is similar. Since 

(Pi,---,Pn) <ox (Pi, ■■■,;£), 

we may, by Strassen's representation theorem, assume without loss of gener- 
ality that 

E((p?,...,^)|(pi,...,ri.)) = (pi,---,pJ.)- 

Theorem 18 together with Theorem 1 imply that for h an increasing convex 
function, 

<JWi,---,K) = EM-Wi^j^),...,^)), 

is a convex function of (p\ , . . . ,p z n ). The rest of the proof goes similarly as the 
proof of comparison lemma 62 with (p\ , . . . , p\ ) substituted for ( A\ , . . . , A\ ) . 

□ 

Also in this setting we can consider the Loynes stochastic variables, as- 
suming that p z n is denned for all n £ Z, 

iWi = i w n {pi n ,...j_ 1 ) 

and 

iWi = lim Ml 

n— »oo 

Assumption 67 A^(j/) is a stationary sequence inn € Z /or i = 1,2. 
With the same proof as in Corollary 7 we then find 

Corollary 8. Under assumption 67, the following holds 

1) (Pl---,PI) <cx {p\,...,pI) for alln>l^ a Wl < icx a Wl- 

2) (q\,...,qi) < cx (qj,...,q 2 n ) for all n > 1 ^ b W^ < icx b Wl. 

11.5 A stochastic lower bound on the traveling times 

In section 11.2 we found that the intuitive argument, that queues in a random 
environment should be bounded below by the corresponding queues where 
the environment is "frozen" to its mean values, is not generally true. As an 
application of the second comparison lemma we will derive in this section 
a lower bound in the icx-ordering. The queueing model is a FSEG with a 
MAP-source. We will construct a more regular (in fact the most regular) 
arrival process with the same arrival intensity as the MAP-source. This will 
provide the lower bound. As we will see this regular arrival process can be ap- 
proximated by a renewal arrival process with Erlang-distributed inter-arrival 
times. Without loss of generality we may assume that the MAP has transi- 
tion times {T n } which form a Poisson(A) process. Let {X n } be the Markov 
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process with transition probabilities X xy which governs the transitions of the 
MAP, i.e. X n is the state at T n . We assume that the Markov process is sta- 
tionary, and we denote by tt x the stationary probability on state x £ E. The 
probability on an arrival at T n is, 

P = ^2^2^xXx y r xy . (11.4) 

x y 

The arrival intensity of the stationary MAP is then A = pX. The MAP 
corresponds to the following admission sequence, 




1 with probability rx n _ 1 x, 
else. 



Since A\ is or 1, hence integer valued for all n, we can also use the 
p-representation, i.e. take p\ = A\, n £ N then 

A n {p 2 )=A 2 n (for all 6). 

Take p 1 = (p,p, . . .) and A\ = A^p 1 ). Then A\ (for fixed 6) is a bracket 
sequence with rate p. It follows from Theorem 20 that the lower bound in icx- 
ordering is obtained if we use the arrival process on T n with A\ as admission 
sequence. Let us call this the Regular Arrival Process with parameters (p, A) 
(RAP(p, A)) . The Markov Arrival Process with stationary distribution tt x , 
x e E and arrival probabilities r xy , x,y € E, we denote by MAP(7r,r). 
Analogous to the /G/l -notation, let us denote the FSEG (with stationary 
sequences T n and S n ) by -/G/SEG and jW 00 (-/G/SEG), i = a,b for the 
potential (i = a) or actual traveling time (i = b) (to a fixed transition). 

Then we have the following theorem, which is an application of compari- 
son Lemma 63. 

Theorem 68. For i = a,b, 

i VF 00 (RAP(| 3 ,A)/G/SEG) < icx i VF 00 (MAP(7r,r)/G/SEG) 

Proof. For i = a we apply part 1) of comparison Lemma 63. Therefore we 
have to show that 

E((p?,...,^)|(pi,...,ri.)) = (pi,...,pi). 

But p\ = p for all k, therefore it suffices to show that E = p, for all k. 
Indeed, this holds since, 

E Pk = ^2 X! n * x *y r *y = P- 

x y 

In order to apply Corollary 8 we have to verify assumption 67. Indeed, the 
admission sequence A\ is stationary since X n is assumed to be stationary. 
Since J* [x + 6\ d0 = x implies 
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E Ai(p) = J([np + 0\- L(n - l)p + 0\)d9 = p, 
o 

it follows that A\{p) is a stationary sequence, and assumption 67 applies for 
both sequences. For i = b, we consider the f^-sequence corresponding to the 
^(p)-sequence. Lemma 21 guarantees that it is a bracket sequence with rate 
1/p. Since A^ip) is stationary, B\ is also stationary. Hence, 

Bi = B n {q) with q = (1/p, 1/p,...). 

Since the A\ sequence is integer valued, also the corresponding B^ sequence 
is integer valued. It remains to verify that EB^ = 1/p, but this is a standard 
result for stationary MAP-processes. rj 

It is well-known that a MAP-process with transition times {T n } = 

Poisson(A) can be represented also as one with transition times {T^} = 
Poisson(.ZVA) with the same it as stationary distribution. If we want to keep 
the arrival intensity equal to A then we have to divide the p and the r xy by 
N, hence 1/p is multiplied by N. Now suppose 1/p is rational, say j^, then 
N 2 /p = Ni is an integer. The corresponding regular arrival process has inter- 
arrival lengths ofiVi steps, hence its inter- arrival times are Erlang distributed 
with JVi phases of exponential-distributed length with parameter N 2 X. 
Using Theorem 68 we will show the following result: 

Theorem 69. For i = a,b, and any real number < c < 1 it holds 

i VF 00 (RAP(p,A)/G/SEG) < icx ^^(RAP^/c, Ac)/G/SEG). 

Proof. RAP(p/c, Ac) can be seen as a MAP with transition times {T n } which 
form a Poisson(A) process. With probability c a transition then is a real transi- 
tion (i.e. a transition of the RAP(p/c, Ac) process). The stationary admission 
sequence in RAP(p/c, Ac) has rate p/c on an arrival at a (real) transition. 
Hence the stationary probability on an arrival at T n is c • p/c = p, and The- 
orem 68 applies. rj 

As a consequence of Theorem 68 we have that for a MAP(7r,r) with p 
(as in 11.4) rational, say p = N 2 /Ni, the FSEG with renewal input with 
JVi phases of exponential distributed length with parameter Af 2 A provides a 
icx-stochastic lower bound on the actual and potential stationary traveling 
times. By Theorem 69 RAP(pc, A/c) for any < c < 1 provides also a 
icx-lower bound. Hence, since RAP(pc, A/c) is arbitrarily close to a renewal 
input for c sufficiently small, we get an approximation for irrational p. These 
facts have been used, without proof, in papers on optimal routing to parallel 
queues (cf. [43]). Clearly the limit-process of RAP(p/c, Ac) for c — ► 00, is 
the renewal-process with constant inter- arrival-time equal to 1/pX (notation 
V(l/pX)). 
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Corollary 9. For i = a,b, 

1) iW 00 (RAP(p/c, Ac)/G/SEG) is monotone decreasing in c. 

2) For any c > 1; 

i W oc {V(l/pX)/G/SEG) < icx i W 00 (RAP(p/c,Ac)/G/SEG) 
<icxi^oo(MAP(7r,r-)/G/SEG). 

Proof. Using a continuity argument (see [67]), the assertions 1) and 2) are 
direct consequences of Theorems 68 and 69. 
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12.1 Introduction 

So far, the notion of multimodularity is related to a particular base, namely 
(T). One may wonder what happens if the multimodular base (J 7 ) is replaced 
by another set of vectors. 

This question is addressed in this chapter, using results from [8], where the 
notion of multimodularity is extended to all possible initial bases. The notion 
of multimodular triangulation was introduced in [27] as a generalization of the 
original concept of atoms in [59] . Here, following [8] , we provide a new sight 
on multimodular triangulations by using a geometrical point of view which 
is simpler (we do not use the set indexing techniques or lower envelops) and 
more general (we do not restrict the triangulations to integer points). From 
an arbitrary vectorial base of W 1 , we show how to build the corresponding 
multimodular atoms partitioning the space and how to define the associated 
simplex convexity property which generalizes multimodularity. 

From there, we define the cones associated with these atoms and the 
cone distance compatible with simplex convexity. Finally, we show how to 
exploit the cone distance to define a partial order relation between arbitrary 
admission policies in G/G/l queues. 

12.1.1 Organization of the chapter 

This chapter is organized as follows. Section 12.2 defines a generalized notion 
of atoms covering the whole space. Section 12.3 shows that starting with a 
multimodular triangulation with simplexes formed by a set of base vectors, 
satisfying simplex convexity is equivalent to the multimodular inequality. 
Section 12.4 shows how one can restrict multimodularity to some sub-spaces 
called sub-meshes. Finally, Section 12.5 defines the cone ordering between bi- 
nary sequences and Section 12.6 provides an application to admission control 
in queues. 
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12.2 Multimodular Triangulations 

Let us start with a matrix D of size (n + 1) x n and of rank n such that the 

rows of matrix D define n+1 vectors (so, ■ ■ ■ , s n ) satisfying so H \-s n = 0. 

Such a matrix will be called a multimodular (m.m.) matrix in the following. 

Any multimodular matrix D can be constructed starting with a n x n 
matrix M with full rank and appending minus the sum of all the rows of M 
as the last row of D. 

Definition 17. The mesh Mp associated with the m.m. matrix D is the set 
of all the points {aoso + a\Si + ■■■+ a n s n , aiGZ, i = 0,---,n}. 

Lemma 64. The following properties are true. 

i) A point in W 1 has a unique non-negative decomposition in (so 
to the addition of (ao ■ ■ ■ a n ) with ao = ■ ■ ■ = a n )■ 

ii) A point in Mp has a unique non-negative decomposition in (so 
to the addition of (ao ■ ■ ■ a n ) with a® = ■ ■ ■ = a n ). 

Proof, i) Since si, ■ ■ ■ ,s n is a base of W 1 , then for any point x in W 1 x = 
otiS\ + ••• + a n s n . for some ^ € 1, i = 0,...,n. Let be the minimal 
coordinate. If < 0, then 

x = Qisi H h a n s n - cti(s H h s n ) 

= -otiSo + (cti - oti)si H h (a„ - oti)s n , ( 12 -1) 

where all the coordinates are non-negative. As for uniqueness, let x = 
ctoso + • • • + ot n s n = floso + • • • + f3 n s n where we may assume that all co- 
ordinates are non- negative and = 0j = 0. If i = j, then a = /3 because 
(so, ■ ■ ■ , s n )\si is a base of W 1 . If no coordinates are jointly null, then we can 

write x = (/3o — /?i)so H \-(0 n — fii)s n which means for the jth coordinate in 

the base (so, ■ ■ ■ ,s n )\si, is ctj = —pi, which is impossible by non-negativity. 

ii) A point in Mp has a unique decomposition in (si, ■ ■ ■ ,s n ), this decom- 
position being in Z. By using the same method as in 12.1, we transform this 
decomposition into a non-negative integer decomposition in (so,Si, ■ ■ ■ ,s n ). 

□ 

Definition 18. A D-atom is a simplex in W 1 , made of the n + 1 points 

Po = a, 

pi = a + s m , 

p 2 = a + Sfto) + (12.2) 

Pn = a + S£( ) + H h 5{(„_l) 

where a € Mp (the root) and £ is a permutation of {0,- • • ,n}. This atom 
will be denoted S(po, ■ ■ ■ ,p n )- 



■■■S n ) (up 
■■■S n ) (up 
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Note that an atom is indeed a simplex since D is of rank n and that 
<S{po,--- ,Pn) and S(pi,--- ,p n ,Po) are two notations for the same atom, 
when starting with p (resp. pi) as a root. 

Definition 19. A collection of simplexes is a triangulation of E (an arbi- 
trary subset ofW 1 ) if 

— E is the union of all the simplexes. 

— The intersection of two simplexes is either empty or a common face. 

Theorem 70. The set of all the D -atoms forms a triangulation ofW 1 , called 
a multimodular triangulation. 

Proof. Let x be a point in W 1 . By Lemma 64, x = otoSo H \-a n s n , with non- 
negative coordinates, one of which is 0. We construct a = [aoJ-soH h|_ a njsn 

and £ such that — |_ a £(i)J > a £(i+i) — l_ a £(i+i)J- We define p n = 0, 
Pn-i = a^-i) - La£(„-i)J, Pi = Q{(i) - L«£(i)J - ( a t(i+i) ~ l a t(i+i)\)> all 

of them verify < pi < 1 and Y17=o Pi < 1- We have x = a + PoS£(o) H 1 - 

Pn-i(s£(o) H hs^(„_i)). Therefore, x belongs to the atom with root a and 

permutation £. 

Now, assume that a point x belongs to the interior of two different atoms 
with respective roots a and b and permutations £ and r. Due to shift invari- 
ance, we may assume that 6 = 0. Moreover, with loss of generality take r to 
be the identity. 

x = a + a sj( ) H h a n -i(s^ ) H h Sf(n-i)) 

= f3 so H h Pn-i(s h 5„_i) 

with Y^=o Pi — 1 an ^ J2^=o a i — 1- Since x is in the interior of both atoms, 
we also have Pi > and a; > for all i = 0, ■■■ ,n — 1. Therefore, by 
uniqueness of the decomposition of x, and writing a = ao^o + h a n ^u, 

a + ^ a £ (j) = /3 H h /?n-i 

n-1 

ai + ^ a £ (j) = Pi H h /3n-i 

n-1 

«n-l + ^ a i{j) = Pn-1 
j^-^n-l) 
n-1 

a n + ^2 a iU) = 0. 
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Since all the partial sums of the oti or of the are all smaller than one 
and since a; are integer numbers, then, a; = for all i = 0, ■ ■ ■ , n. Both 
atoms have the same root. 

Now, the equality of the partial sums taken one by one imply first that 
Sj=£- 1 (n) a (,U) = 0- Since ct; > for all i, then the only possibility is 
£ _1 (n) = n. Considering vectors Sk and Sk+i, we have: 

71 — 1 71 — 1 

X! a su) - X! a (U) = fa 

i=«" 1 (fc) i={- 1 (fc+i) 

> 0. 

This implies that £ -1 (fc) < £ _1 (fc + !)• This means that £ is the identical 
permutation. Therefore, both atoms are equal. □ 

In the restricted case when the mesh is TL n and D the incidence matrix of 
a graph, this theorem was proved in [27] using an intricate argument of set 
indexing (see the example presented in Figure 12.1). 

Lemma 65. Combinatorial properties of m.m. triangulations. 

i) A point in M D belongs to (n + 1)! D-atoms. 

ii) The unit-cube in Mp is partitioned into n\ D-atoms. 

Proof, i) This is a straightforward consequence of the definition of atoms. 

ii) The unit-cube U in Mp is the set of points of the form a\S\ H h a n s n , 

with ai e {0, 1} for all 1 < i < n. Given a permutation £, there exists a point 
b £ U such that the atom S(b, b + s^ ) , " ' ' > b + s^-i) is included in U. The 
point b = b\S\ H h b n s n is chosen in the following way: 



JO if 
\1 o1 



r^xrno), (123) 

otherwise. 



Each atom in U has n + 1 vertices, hence n + 1 representations of the form 
S(b,b + 5^(0),- •• ,b + S£( n _ij). Finally, combining this with part i), (n + 
l)!/(n + 1) atoms are contained in U. Since all the atoms triangulate W 1 , 
those in U triangulate U. □ 

Some multimodular triangulations have a special interest. The most used 
ones are the triangulations with a m.m. matrix D being the incidence matrix 
of a graph. 

An oriented tree is a graph G = (V, E) with n + 1 nodes and n arcs. It 
has an incidence matrix D of size (n + 1) x n defined by 

(+1 if vertex i is the start point of edge j 
— 1 if vertex i is the end point of edge j 
otherwise 
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First note that whenever D has rank n, then the graph has to be a tree. 
Also, since D is totally unimodular, then its mesh Md is 1 n (see for example 
[54] for a detailed presentation on totally unimodular matrices). 

The L-triangulation is the triangulation associated with a linear graph 
(see Figure 12.1 for an illustration in M 2 ). The associated m.m. matrix is 



D 



( 1 
-1 1 
-1 1 







\ 





-1 1 

-1/ 



(12.4) 



The tree associated with the 
L-triangulation in R 2 . 




The atoms of the 
L-triangulation in R 2 



Fig. 12.1. The L-triangulation in dimension 2 



On the other hand, some triangulations of the space into simplexes are 
not multimodular triangulations. Such examples in dimension 2 and 3 are 
given in Figure 12.2. The simplexes in the triangulation displayed in 12.2(a) 
use 4 different base vectors, si,fi2,S3,S4- The triangulation in 12.2(b) de- 
composes the unit cube in 5 simplexes instead of 3!=6 for any multimodular 
triangulation. 



12.3 Multimodular Functions 

Let D be a m.m. matrix with row vectors, (so,--- ,s n )- Sometimes in the 
following, the reference to D may be omitted. Everything implicitly refers to 
D such as multimodularity and atoms. 

Definition 20. A function f : Mp — * ffi is D -multimodular if and only if 
for all a £ Mp, and for all < i < j < n, 
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(a) Triangulation of R 2 (b) Triangulation of the cube in R 3 

which is not multimodular into a minimal number of simplices. 

It is not multimodular 
Fig. 12.2. Triangulations of R 2 and R 3 which are not multimodular triangulations 



f(a + Si) + f(a + 8j ) > f(a) + f(a + Si + Sj). (12.5) 

From f , we construct a function f : W 1 — > M by linear interpolation of f over 
the atoms defined by D. 

Note that / depends on D. 

Theorem 71. / is D -multimodular if and only if f is convex. 

The following proof is completely different from the former proofs and 
gives a generalization to more general meshes. The hard part ("only if"), in 
the restricted case of the L-triangulation, was done in [59]. The extension to 
the more general case of tree triangulations as well as the easy reverse part 
("if" part) were presented in [27]. A version of the proof presented here can 
be found for the restricted case of the L-triangulation in Chapter 1. 

A function / which is .D-multimodular may also be called simplex convex, 
considering Theorem 71. 

Proof, "only if" : The function / is continuous by definition. Moreover, along 
any direction 6, it has only a discrete number of isolated points where it is 
not differentiable. By using the characterization of convexity given in [96], 
we will check convexity at a point z by showing that for point z, and any 
direction 6, the right derivative is greater than or equal to the left derivative. 
It obviously suffices to check at points that are on the boundary of an atom, 
since, by definition, / is linear in the interior of atoms. 



12.3 Multimodular Functions 249 



Hence, we first assume that the point z is on the interior of a face (of 
dimension n — 1) which is common between two adjacent atoms. Without 
loss of generality, assume that the atoms (defined below by their extreme 
points) are 

S = S(xq,xi, ...,x m ) and S = S(x ,xl, ...,x m ). 
where x, satisfy (12.2) and 

X^ = Xq -\- Si 2 , X2 — X± -\- . 

Decompose direction 8 in its projection <5 2 in the common face between 
the two atoms and in the component 81 along the direction (x\ — X\). In 
the direction 82, the left and right derivatives are equal. In the direction Si, 
the right^ derivative is a constant c, depending on the length of Si, times 
f{x\) — f(z). The left derivative is c(f(z) — f{x\)). Omitting the constant c, 
and using point z = \(x$ + x%) hence 2f(z) = f(xo) + f{%2), we get for the 
difference 

(7(*n- /(*))-(/(*)- /to)) 

= (f(xl)-f(x ))-(f(x 2 )-f( Xl )) (12.6) 

The fact that (12.6) is nonnegative follows by applying 12.5 with x = Xq, 
and 

X^ — Xq ~\~ Si 2 

Xl = Xo + Si 2 + Si 1 

£l = #0 + Sij. 

It now remains to consider the case where the direction S in point z crosses 
from atom S to atom S, and S n S is of dimension at most m — 2. In that 
case, we consider the cylinder C in direction 6 containing the hyper-face F 
of S, opposite of z in direction S. 

The intersection of C with an arbitrary atom A is of dimension, say k and 
its projection on F along direction S is the intersection of the projections of 
C and A and has dimension at most k. Therefore, F is almost everywhere 
(in Lebesgue measure) covered with such projections of dimension m — 1. 
Therefore, we can find an affine line L with direction S, included in C, that 
intersects S and S, the projection of which is a point in F not belonging to 
intersections of dimensions smaller than m — 1. Therefore, we can claim that 
L only intersects faces of atoms of dimension m — 1. The convexity in point 
z and direction 8 now follows from the convexity in points z, corresponding 
to the intersections of line L with all the intermediate atoms between S and 
the last atom, S. 

This construction is illustrated for dimension 2 by Figure 12.3. 
"if" : Consider an arbitrary point Xo and any two distinct elements Si , Sj in 
T. We have to show that 
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Fig. 12.3. Construction of cylinder C and line L in dimension 2. 



/(x ) + /(x 2 ) - /(an) - /(xj) < 0, (12.7) 

where xi = xo + Si, xj = xo + Sj, x 2 = xi + = xj + s^. 

Define z = \{x\ + x\) = ^(xo + X2) and consider the line segment xi — ► 
z — ► xj. The left derivative (l.d.) and right derivative (r.d.) in z are given by 

= /(^(X! +Xl)) - J{ Xl ) = i/(x ) + i/(x 2 ) - /(Xi), 

r.d. = J{xl) - /(i(x + x 2 )) = /(xj) - i/(x ) - \f{x 2 ). 

Since / is convex, r.d. — l.d. is non-negative, and hence (12.7) holds. rj 

Lemma 66. We consider all the n-periodic sequences a = (a;); G N with val- 
ues in N satisfying 

n 

Y^ai = k. (12.8) 
i=i 

Let f be a multimodular function defined on a mesh Mp of dimension n. 
Then, the quantity 

n 

^2 f( a i S l H a 

2+71 — 1 $11 

i=i 

is minimized at point r = {^s\ + h ^s n ). 

Proof. Let P be the set of all integer sequences a, which are n-periodic and 
such that within one period, they add up to k, Yl7=i a i = k- By periodicity, 
and using Jensen inequality, 

1 n k k 
min - V] /(ajSi H h a i+n _is„) = nf{-si H h -«„). 

2=1 



□ 
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12.4 Sub- meshes 

Definition 21. A sub-mesh P ofMp is the intersection ofMp with a convex 
set ofW 1 which is the union of (faces of) D-atoms. 

Since any union of -D-atoms which forms a convex set is a sub-mesh 
by definition, typical sub-meshes are: the positive quadrant: {ao^o + ■ ■ ■ + 
a>n-iSn-i,a>i € N} and the unit cube {aoSo H + a-n-iSn-i, a i G {0, 1}}. 

The hyper-plane {aofio H ha n -is n -i, J2i a i = k,ai G Z} is a sub-mesh 

of dimension n — 1 of the L-triangulation. An example of such a sub-mesh in 
dimension 3, P = {x + y + z = 2,x,y,zG N}, is given in Figure 12.4, 



z 




Lemma 67. Any sub-mesh P has the following properties: 

i) The faces defining P form a multimodular triangulation of P. 

ii) The vectors of this new multimodular triangulation are disjoint sums of 
the original vectors. 

Proof, i) First, by convexity, P has the same dimension, say k, as any of its 
faces. It should be obvious that the faces defining P form a triangulation of 
P. Now, let us show that this is a multimodular triangulation. 

Let F 1 (resp. F 2 ) be a face in P of a simplex S 1 (resp. S 2 ) with root a 
(resp. b) and permutation £ (resp. 7). Without loss of generality, we may as- 
sume that a e S 1 and b € S 2 by shifting the starting points in the definitions 
of atoms S l and S 2 . 
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Now, the vertices of F 1 (resp. F 2 ) are visited in an order depending on £ 
(resp. 7), such that 

pl=a Po = b 

p\ = a + s m + 1" Pi = b + s 7 ( ) + H -s 7 (ii-i) 

P\ = Pi + + • • • + 5{(i a -l) Pl=Pl+ S 7 ( h ) + ■■■ + S 7 (j a -1) 

Since both faces are in the same space of dimension k, then for any to < k, 
we have the linear combination 

fc-i 

s i(j m ) + l- s 7(i m +i-i) = $^aws£(ii) + 1" 

£=0 

This is a linear combination between the vectors so,si,--- ,s n (with the 
convention that Iq = and jo = 0). We know that the only relation is 

•So + S\ H h s n = 0. Therefore, for all to, there exists a unique q such that 

Oi m , q = 1- (all the other coefficients a m> i are null). 

S l(j m ) + ■■■ + « 7 (j™+i-l) = + • • • + 

This means that the vectors defining face F 2 are the same as the vectors 
defining face F 1 (up to a permutation). q 

Lemma 68. Let p be a point in Mp, £ be a permutation of {0, • • • , n} and 
{ik}k=o,---m be an increasing sequence of integers with io = and i m = n. 
We define the vectors Sj = + ■ ■ ■ S£(i j+1 -i) for all j = 0, • • • to. The set 
p+ {clqSq H h a m s m ,ai € N} is a sub-mesh of Mp- 

Proof. First note that the set P is a convex subset of Mp of dimension to. 
Now we consider the triangulation of P defined by the vectors so , ■ ■ ■ ,s m , 
each atom of this triangulation is a face of a .D-atom. q 

Thus the hyper-plane P = {aofio + ■■■ + a n -is n -i , J2i a i = fc} is a 
sub-mesh of dimension n — 1 of the L-triangulation by choosing s = 
81, ■ ■ ■ ,s n - 2 = s n -i and s n -i = s n + s , which yields P = (k, 0, ■ ■ ■ , 0) + 
{x s H h x n -is n -i,Xi e N}. 

Lemma 69. A function f which is D-multimodular is multimodular on any 
sub-mesh P with respect to the induced multimodular matrix on P. 

Proof. Let a be a point in P and let u and v be two arbitrary rows for 
the multimodular matrix of P. By Lemma 67, u = + ■ ■ ■ + Si k and v = 
s h + • • • + Sj m where the sets {si 1 , • • • , Si k } and {sj 1 , • • • , Sj m } are pairwise 
distinct. Therefore, we have 
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f(a) + f(a + u + v) = f(a) + f(a + s il -\ h s ik + s h H h s jm ) 

< f(a + s n -\ h s ik ) + f(a + s h + h s jm ) 

= f(a + u) + f(a + v). 

□ 

Corollary 10. The function f is multimodular in a sub-mesh P if and only 
if f is convex on P, the convex hull of P. 

Proof. It should be clear from the proof of Theorem 71 that the equivalence 
of the multimodularity of / and the convexity of / still holds if we restrict 
the function / to P. □ 

A second corollary of Theorem 71 concerns the minimization of multi- 
modular functions. For a function defined on P, we call x a local minimum 
on P if f(x) < f(x + SiSi) (with £; £ {—1, 1}) for all i such that x + SiSi is 
in P. 

Corollary 11. Let the function f be multimodular in P. Then a local mini- 
mum is a global minimum on P. 

Proof. If / is multimodular in P, then / is convex in P, and is linear on the 
(faces of) atoms forming P. The graph of / (i.e. {x : 3y s.t. x > f(y)}) is a 
convex polytope. Therefore, all the local minima are global and are extreme 
points of atoms. □ 



12.5 Cones 

Now, the convex space P of dimension n will be divided into (n + 1)! cones, 
all starting at point h, any point of the mesh of P. Consider one atom 
S = S(po,pi, ■ ■ ■ ,p n ) containing h = po as a vertex. Let £ and r be the 
permutations on {0, • • • n} such that r(0) = and 

Pr(i) = h + s m (12.10) 

Pt(2) = Pr(l)+H(l) (12.11) 

: = : (12.12) 

P T (n) = Pr(n-l) + ««(n-l) (12.13) 
h = Pr(n) +««(n)- (12.14) 

Now, we define for all 1 < i < n, bi = Y?j=i s £(j) an ^ b = (61, ■ • • , b n ). 
Therefore, p T ^ = h+bi. The vectors (61 , • • • , b n ) will be called the generators 
of the cone. 

The cone associated with S, denoted C(S) is made of all the points p of 
P such that p = h + c t b where c is a non-negative vector in M 1 . First, note 
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that, for each p in the cone, vector c* is uniquely defined since (61, ■ ■ ■ , b n ) 
are independent vectors. Second, note that when we consider all the atoms 
containing h as a vertex, then all the (n + 1)! associated cones will cover P. 
If two adjacent atoms share a face, the two corresponding adjacent cones will 
also share a "face" (of dimension n — 1). 

For any point p in P, p will be in at least one cone and we will have 
p = h + c t (p)b, where b is uniquely defined on the support of c(p). note that 
p can be on the boundary of several adjacent cones. 

We shall denote d(h,p) = c\(p) H \-c n (p) and call it the cone-distance 

from h to p. 

All the previous remarks show that d(h,p) is well defined. 
12.5.1 Minimization 

In this section, we consider the case where P = W 1 , where P is the convex 
hull of P, and h = 0. We also consider a function / multimodular with respect 
to the row vectors of D, so, ■ ■ ■ ,s n . 

We focus on one arbitrary cone, C, defined by the permutation 7. This 
means that the generators of C are the vectors 

bo = S 7 (0): 

h = s 7 (o) + 57(1), 

bn-l = 57(0) + 57(1) H 1" 5 7(n-l)- 

Any point in C has non- negative coordinates (3o,--- ,/3 n -i in the base 
bo,-- - , b n -i- 

We call # the linear transformation which is the passage from the base 
bo,-- - ,b n -i to the base s 7 (o), s 7 (i), ■ ■ ■ , s 7 (n-i)- 

Lemma 70. Lei fc 6e an integer. The set Ck of points in C such that (io + 
• • • + Pn-i = k is a sub-mesh of Mp. 

Proof Set p = fcs 7 ( ) and ~s = s 7 (i),-" ,s n -2 = s 7 (n-i), «n-i = 57(0) + 
s 7 ( n ) • Then the sub-mesh p + a^s^ , a; € N} n C is precisely the set Ck ■ □ 

The following lemma is some kind of generalization of Theorem 6 from the 
L-triangulation and the positive quadrant to any multimodular triangulation 
and one of its cones. 

Lemma 71. Let f be a m.m. function, then the quantity 

1 " 

-£/<>*(#,... ,/3 i+n _i) 
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is minimized over the set Ck at all the points (3(8), < 8 < 1 of coordinates 
p i (8) = [i^ + 8\-[(i-l)^ + 8\. 



Proof. Using Corollary 10, the function ^ J27=i f ° ^(ft> " ' > ft+n-i) is con- 
vex. Moreover, it is minimized at point r = (/3o = ■ ■ ■ , Ai-i = over Cfc 
(see Lemma 66). 

This function is linear on the atoms of the sub-mesh Ck- Therefore, it 
is also minimum at all the vertices of an atom containing the point r. The 
vertices of this atom contain the points j3{8) with coordinates 

m = [(i + l)^- + 8\-[i^ + 8\, 

when 8 varies from to 1. 

To show this, first note that these points are all in the sub-mesh, since all 
their coordinates add up to k and since &(f3(8)) is integer valued. 

Second, let fi = 1 — (i + 1)£ + [(i + 1)^J , ordered in the increasing order. 

By construction, when 8 varies from to 1, then the points j3{8) have at 
most n — 1 values. The point ft (6) changes at all the points of the form fi for 
i = 0, • • • , n — 2. At 8 = fi we add — s,+i (if fi = fj then when 8 = fi we add 
—Si+i — Sj+i). Therefore, all the points j3{8) form a face of an atom. Noting 
that 

r = + -^—P(f ) + -^-rP(h) + ■■■ + -^—(3(fn-2), 

n — 1 n — 1 n — 1 n — 1 

shows that r belongs to that face. rj 



12.5.2 Cone ordering and monotonicity 

Now, we define a partial order on P by choosing h = r. We only consider 
multimodular functions for which r is a minimal point. 

First, this partial order (called cone ordering) is defined in a different 
manner on each cone. 

In a given cone C, with generating vectors &i, ■ ■ ■ ,b n ,we say that x <c y 
if c(x) < c(y) component-wise. Note that a cone with this partial order is a 
lattice which is isomorphic to N 71 with the classical component-wise order. 

Theorem 72. If f is a multimodular function on P, then x <c y implies 
that f(x) < f(y). In other words, f is monotone with respect to the partial 
order <c- 
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Proof. Since x and y are comparable, this means that they are in the same 
cone. From now on, 61, ■ ■ ■ ,b n will be the generators of this cone. 

First note that we can assume that d(x, y) = 1. If not, then we prove step 
by step along the path from x to y along the direction of the generators, say 
x = x 1 < c ■ ■ ■ <c x m = y that f(x) = f{xi) <■■■< f(x m ) = f(y). 

The proof will now proceed by induction on d(r, x) . 

First note that the property is true if d(x,r) = 0, since r is the argmin of 
/ on S. 

Now, let us assume that we have d(x,r) > 1. Pick a point z such that 
x = z+bi in cone C. Equivalently, we have c(y)+ei = c(x) and c(y) < 0. Note 
that d(z,r) = d(x,r) — 1 and z <c x. By induction, this means f(z) < f(x). 

Since d(x,y) = 1, there exist j such that y = x + bj. Now we have two 
cases, since we may not be able to choose i such that i = j. 

— If i = j, then by convexity of /, 

f(z + bi) - f(z) < f(z + bi + bi) - f(z + bi). 

We also know by induction that f(z + bi) — f(z) > 0. This means that 
f(x)<f(y). 

— If i ^ j, then we choose yet another point, w, such that w = z + bj. We can 
assume that i > j (the case j < i is similar by inverting the role played by 
bi and bj in the following). 

Since bi is a sum of base vectors, it is also a sum of opposites of base 
vectors, since all base vectors add up to 0. Note that all these base vectors 
are distinct from the base vectors involved in bj. 
We have: 

h = s m + ■ ■ ■ + S £U)' ( 12 - 15 ) 

bi = - Sf(i+2) - • • • - S£( n+ iy (12.16) 

Therefore, 

f(w) - f(z) = f(x + s e{1) + ■■■ + s m + s ({i+1) + ■■■ + s e{n+1) ) 

-f(x + S^ i+1 ) + hS£(„+l)) 

< fix + fl £( i) + • • • + s m ) - f{x), (12.17) 
= fiv) ~ fix)- 

where Inequality 12.17 is a direct consequence of the definition of multi- 
modularity. Since d(r,w) = d(r,z) + l, by induction we have, f(w) — f(z) > 
0, then this implies f(y) — fix) > 0. 



□ 
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12.6 Application: periodic admission sequences in 
G/G/l/oo tandem queues 

We consider queues in tandem with general stationary service times. As for 
the arrival sequence, let (v,i)i£N be a stationary process. The integer sequence 
{ai}i G N is the admission sequence into the queues. The interarrival times of 
customers in the queue is a sequence (Tj)j 6 N defined by: 

aiH hai 

j=aiH hai_i 

The construction of the inter-arrival sequence is illustrated in Figure 12.5. 

(i.i.d sequence) 
] 



ri T2 T3 (inter-arrival time) 

ai — 2 ct2 — 2 as — 2 (admission sequence) 

Fig. 12.5. The construction of the inter-arrival sequence from a sequence u and 
admission a = (2, 2, 2, • • • ). 

In the following, the admission sequence will be assumed to be periodic 
with period n. 

As to introduce multimodularity, we choose the L-triangulation of Z™. 
The atoms given by the -L-triangulation with row vectors Si = —ti + e^+i, 
and s =ei,s„ = — e n , as in 12.4. 

The sub-set of 1 n that we will work with is P = {(do,-- - ,a n ),cii > 
Vi, J27=o ai = k}> where k is a given integer. 

The set P corresponds to all admission sequences with k admitted cus- 
tomers among n slots. 

Lemma 72. P is a convex union of hyper-faces of atoms in 1 n . 
Proof. Let us consider the constraints one by one. 

— The constraints aj > restrict P to N n which is made of a convex union 
of atoms. 

— Now, let us look at the constraint X^=o ai = ^- ^ ms constraint is a convex 
union of faces of atoms. 

To finish the proof, remark that the intersection of convex union of faces 
of atoms is a convex union of faces of atoms. n 
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The atoms on P are defined by the vectors s[ = si, ■ ■ ■ , s' n _ 1 = s n -i as 
for the L-triangulation of Z n , and a new vector s' = so + s n = e n — e\. 

If / is a multimodular function, / : Z 71 — > ffi, we will consider the re- 
striction of / to P which is also multimodular on P with its own atoms (see 
Lemma 69). By Corollary 11 / has a global minimum on P. In the following, 
this minimum will be called r. 

Theorem 73. The average expected waiting time in a stochastic event graph 
is a multimodular function on P. 

Proof. From the vector a = (ai,--- ,a n ) we construct an infinite sequence 

clef 

a = a u . Let #jv(ai,--- ,a„) = ^ J2k=i Wfc(ai, • • • ,a k ), where W k is the 

def 

expected total sojourn time of the fcth customer, and let G(ai, ■ ■ ■ ,a n ) = 
linijv^oo 9jv(«i, ■ • • , a„). We also denote p the largest integer such that pn < 
N. 

From Theorem 19, we know that Wk is multimodular with respect to 
the L-triangulation in Z fc . Since the m.m. matrix for the -L-triangulation 
in Z fc is a sub-matrix of the m.m. matrix for the L-triangulation in 
then we also know that Wk is multimodular in Z N . Therefore, the function 

H(ai, ■ ■ ■ ,Qjv) jf J2k=i w k{o-i, ■ ■ ■ ,otk) is multimodular in Z w . 

For all < i < n, g^ia + Si) = H(a + Si + Si +n + • • • + Si+kn), where 
k = L^r 1 ] • Now, using the general characterization of multimodularity, that 
is f(a + S\ + S2) — f{a + S\) < f(a + S2) — /(«), for Si and ^2 any arbitrary 
sum of base vectors, with the only restriction that no base vector appears in 
Si and in S2 , then it is immediate to check that gN{a + Si + Sj) — gN{a + Si) < 
9n(o + Sj) — gN{d), for Si and Sj any arbitrary distinct m.m. row vectors. 

Therefore, g^ is multimodular in 1 n . The limit G is also multimodular 
in 1 n . By using Corollary 69 , G is also multimodular on P. q 

12.6.1 An example 

We want to compare the expected waiting time under admission sequence 
a = ( 1 , 2 , 3) and under admission sequence b = (1,1,4). 

The corresponding space P is the set {(x, y,z),x + y + z = 6}. The space 
P is of dimension 2 with induced multimodular vectors Si = (+1, — 1,0), «2 = 
(0,-1-1,-1) and s = (-1,0, +1) (from the L-triangulation of M 3 ). 

The cost function G (expected waiting time) is minimized at point r = 
(2,2,2). This is a direct consequence of a combination of Lemma 71 (used 
with the L-triangulation and the cone constructed with the n first vectors, 
which is the positive quadrant) and Theorem 73. 

If we consider the cone Ci generated by 61 = so and 62 = so + si, then 
we have a = r + bi and b = r + bi + 62. Therefore, a <c b, which implies 
G(a) < G(b) by Theorem 72. This example is illustrated in Figure 12.6. 




Fig. 12.6. comparison of b = (1, 1, 4) and o = (1, 2, 3) 
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13.1 Introduction 

In Chapter 12 we studied the question which of two deterministic periodic 
admission sequences (periodic sequences of nonnegative integers) gives the 
smaller average expected waiting time. A partial order, called the cone or- 
der, is introduced there, and it is shown that the average waiting time and 
more generally any multimodular function is monotone with respect to the 
cone order. It is natural to define a multimodular order by requiring that 
any multimodular function is monotone. In contrast to Chapter 11, where 
we used the convex order for stochastic admission sequences, we consider 
deterministic admission sequences in this chapter. Note that deterministic 
admission sequences can only be ordered for all multimodular functions if 
they are equal. Therefore we consider multimodular functions with a fixed 
minimal point as we did with the cone order. In Section 13.2 we introduce 
the multimodular order and we show that the cone order is equivalent to 
the multimodular order. In Section 13.2.1 the shift invariant counterparts of 
these orders are studied and it is shown that the regular admission sequence 
is the minimal point. We introduce a combinatorial notion which we call the 
unbalance of the admission sequence in Section 13.3. Roughly speaking it 
measures its 'distance' to the regular admission sequence. We show that the 
unbalance is a shift invariant and multimodular function. For the definition of 
the unbalance we introduce in Section 13.3 the graph order, which gives the 
distance of a given admission sequence to the bracket sequence and thereby 
defines its unbalance. The relations between the shift invariant orders and the 
graph order are studied in Section 13.4. We show through counterexamples 
that they are not equivalent. 

For the optimal routing problem to n queues we derived in theorem 26 that 
a lower bound is obtained by using regular admission sequences (or what is 
the same bracket sequences) with 'minimizing' routing fractions (densities) 
as admission sequences to all queues. But generally only in the routing to 
n = 2 queues, the routing fractions will be balanceable (see section 6.2) and 
only in that case the admission sequences can be glued together and be made 
to a feasible routing policy. In Section 13.5 we derive an upper bound for the 
difference between the average expected waiting time for customers which 
are routed to one queue with a given periodic admission sequence which has 
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a fixed routing density, and the lower bound corresponding to the regular 
admission sequence with the same density. In Section 13.6 we derive then an 
upperbound for the average expected waiting time for a routing policy to n 
queues as a function of its unbalance. For a routing policy the unbalance is 
the sum of the unbalances of the induced admission sequences to the queues 

1,...,TJ. 

The difference between the upper and lower bound depends on the average 
interarrival times but it is insensitive for their distributions and the service 
time distributions. The upper bound is tight for the heavy traffic situation 
in which the interarrival and service times are deterministic and the traffic 
load is 1 for each of the queues. This means that the upper bound is correct 
for the deterministic heavy traffic case. Clearly, it holds for low traffic that 
the average expected waiting time is close to the lower bound. 

All this is mainly based on the unbalance which is a combinatorial notion 
and the analysis in all this chapter is mainly combinatorial, it turns out 
that the unbalance is also useful in other models than routing to queues. 
Indeed, the bounds can be generalized from waiting times to similar bounds 
for sequences of multimodular functions as in Section 1.3 (see the remark at 
the end of Section 13.4). This chapter is a modified version of [49]. It uses 
at some places the results of the technical report [68]. A shortened version 
of this report appeared as [69] and an extended version will be published in 
two parts [72] and [71]. 

13.2 The multimodular order and the cone order 

Let D be a multimodular matrix with rowvectors do , d\ , . . . , d n £ W 1 satis- 
fying do + ■ ■ ■ + d n =0 and let Mp C W 1 be the corresponding mesh. Let 
r e M D and let 

T{D,r) = {/ : M D -> M s. t. / is D - multimodular and f(r) = min f(x)} 

xEM D 

be the set of D - multimodular functions with global minimum point in r. For 
given multimodular matrix D and point r £ Mp we will use the multimodular 
order < mm and a cone order <c on Mp. The multimodular order is defined 
below and the cone order is defined in Section 12.5. 

Definition 74 Let the multimodular matrix D and global minimum point 
r £ Md be given. Then for x,y € Mp we say that x < mm y if f(x) < f(y) 
for every f € F(D,r). 

It can be shown that both the multimodular order < mm and the cone 
order <c are reflexive, antisymmetric and transitive and thus they are partial 
orders on Mp. The following theorem says that these partial orders on Md 
are equivalent. 
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Theorem 75 For given multimodular matrix D and global minimum point 
r G Md we have for x, y G Md that x < mm y if and only if x <c y. 

In Theorem 72 it is shown for x,y G Md that x <c y implies that 
x <mm y- Thus to prove Theorem 75 we have to show that x < mm y implies 
that x <c y- By translation we can assume without loss of generality that 
that the global minimum point r = (0, 0, . . . , 0) G M D . Let C = {d}^ 1 ^ 
be the set of cones corresponding to D and root r = (0,0, ... ,0) (the origin) 
as denned in Chapter 12 . Then {uC; : d G C} = W 1 and for d G C let C[ 
be the intersection of Ci and M D - We have the following proposition ( see 
Lemma 64 and Section 12.5). 

Proposition 18. Let Ci G C. Then there exist b\, 62, ■ ■ ■ , b n G ffi™ and a 
bisection a : {1, 2, . . . ,n + 1} — ► {0, 1, . . . ,71} such that 

1. bi, 62, ■ ■ ■ , b n are linearly independent. 

2 - h j = Ei=i d °(i) f° r j = 1, 2, . . . , n. 

5 - ^ = {ELi Ai ■ 6i : Ai > /or t = 1,2, . . . ,n}. 

4. M D = {E7=i *i ■ bi :Xi G Z /or i = 1,2, . . . ,n}. 

5- C? = {Efci Ai • 6i : Ai e Z> /or i = 1,2, . . . 

Remark 28. For a convex cone C C M" we say that a set of vectors 
vi,V2, . . . ,Vk are generators of C if v\ , V2 , . . . , Vk are linearly independent 
and C = {Ei=i Ai • Vi : Ai > for i = 1, 2, . . . , k }. In that case we say 
that C is a fc - dimensional cone. In particular we have for 61, 6 2 , ■ ■ ■ , b n as 
in Proposition 18 that they are generators of the n -dimensional convex cone 
d- 

Let &i, 62, ■ ■ ■ , b n be generators of cone Ci € C as in Proposition 18. If x € 
then we have by Proposition 18 that there exist unique Xi € Z such that 
x = E"=i x « " F° r ^ = 1, 2, . . . , n we define the function fk : M D — * Z> 
by 

/ fc (x) = max(x fc ,0). (13.1) 

For i = 1, 2, . . . , (n + 1)! let 6(d) be the set of functions {/1, /2, ■ ■ ■ , /«} 
for cone Ci and let £? = ^"^'(/(Ci). For Ci € C with generators 61, 62, ■ ■ ■ , b„ 
as in Proposition 18 we say for x = E"=i %i ■ bi € W 1 that suppj(x) = {fc € 
{1,2, . . . ,71} for which Xk > 0}. Thus fc G suppj(x) if and only if fk(x) > 
for fk G 6(Ci). Note that we assume for k G suppj(x) not only that Xk 7^ 
but that Xfc > which is stronger than in the standard definition of support. 

We have the following proposition. 

Proposition 19. For given multimodular matrix D and global minimum 
point r = (0,0,..., 0) G M™ we have that 6 C T(D,r). 

Proof. For / G 6 we have that / = fk G <7(Ci) for some fc G {1, 2, . . . , n} and 
i G {1, 2, . . . , (n + 1)!}. Let &i, 62, ■ ■ ■ , b n be the generators of cone Ci as in 
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Proposition 18. Then for x = J27=i x i-bi £ we have that f(x) = fk{x) = 
max(xfc, 0). Since f(x) > for every x £ Mp and f(r) = it follows that r is 
a global minimum point of /. It remains to prove that / is a D -multimodular 
function. So, we have to show that for every x £ M D , i,j £ {0,1, ... ,n} with 
i ^ j it holds that 

f(x + di) + f(x + dj) > f(x) + f(x + di + dj). (13.2) 

According to property 2 of Proposition 18 there exists a bijection a : 
{1,2 . . . ,rc + 1} -> {0, l,2,...,n} such that bj = J2i=id a {i) for j = 
1,2,... ,n. From this it follows that <2 CT (i) = h, d a ^ = bj — bj-i for 2 < j < n 
and = - Yn=i d a(i) = -b n . Put / = a(k), to = a(k + 1). If i $ {I, m} 

and j $ {I, to} then we have for every x € Mp that f(x + di) = f(x + dj) = 
f(x) = f(x + di + dj). Hence (13.2) holds . Suppose that i € {l,m} and 
j $ {l,m}. Then we have for every x € Mp that f(x + di) = f(x + di + dj) 
and f(x + dj) = f(x) and thus (13.2) holds. Suppose that {i, j} = {l,m} and 
assume without loss of generality that i = I and j = to. If f(x) > then it 
follows that f(x + di) = f(x) + l, f{x + dj) = f(x)-l , f{x + di + dj) = f(x) 
and thus (13.2) holds. If f(x) = then it follows that f(x + di) > f(x), 
f(x + dj) = f(x + di + dj) = f(x) = and thus (13.2) holds. So, we can 
conclude that (13.2) holds in every case and thus / € ^(D^). rj 

Proposition 19 has the following corollaries. 

Corollary 76 If x,y € Mp, x < m m V and there exists some cone d € C 
such that x,y £ Ci then x <c y- 

Proof. Let b\, bi, . . . , b n be the generators of cone Ci as in proposition 18 and 
put x = Y17=i x i'bi and y = J27=i Vi'bi- Then we have by Proposition 18 that 
Xi,yi e Z> for i = 1, 2, . . . ,n. So, for f k £ Q(Ci) we have that fk(%) = Xk 
and fk{y) = yk for k = 1,2,..., n. Hence by Proposition 19 and x < mm y we 
have that < x k = fk(x) < fk(y) = yk for k = 1, 2, . . . , n. Thus x < c y- □ 

Corollary 77 If x,y e M D , x < mm y then suppj(x) C supply) for i = 
l,2,...,(n + l)!. 

Corollary 77 can be proved in the same way as Corollary 76. The following 
corollary follows immediately from Corollary 76 and Corollary 77. 

Corollary 78 If x,y € Mp, x < m m y and there exists some cone Ci € C 
such that x is an internal point of Ci then x <c y- 

To prove Theorem 75 it suffices by Corollary 76 to prove that there exists 
some cone Ci £ C such that x, y £ Ci if x < mm y.Ifx = r,y = roiy = \- x 
with A > then it follows directly that x,y £ Ci for some Ci £ C. Suppose 
that y = X ■ x with A < and x ^ r. Let Ci £ C be a cone containing x 
with generators b\, bi, . . . , b n . Then x = J27=i Xi " ^ anc ^ V ~ S™=i ^ " Xi " ^ 
with Xi > for i = l,2,...,n. Moreover we have that Xk > for some 
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k £ {1,2, ... ,n} since x ^ r. Then for fu € Q{Ci) we have that fk{x) = 
Xk > and fk{y) = 0, which contradicts x < m m V- So, we can assume that 
x and y are linearly independent vectors in W 1 and let H = span(x,y) the 
two dimensional subspace that contains x and y. For C; £ C let C* be the 
intersection of Ci and if. Then we have that the C* are convex cones of 
dimension smaller or equal than two in the subspace H. To prove Theorem 
75 we have the following lemmas. 

Lemma 73. Suppose that C* = CiflH is a two dimensional cone generated 
by u and v. Then supp^w) is not a subset of supp^v) and suppj(i;) is not a 
subset o/supp^w). 

Proof. Let b\, b 2 , ■ ■ ■ , b n be the generators of Ci as in proposition 18. Then 
u = Y,ke S u PPi (u) u k ■ h with u k > and v = Eke S u PPi (v) v k ■ h with v k > 0. 
For a small positive number e we have that w := v — e ■ u € H and if 
suppj(w) C suppj(t;) then it is easily seen that w = J2kesu PP (v) w k ' bk with 
Wk > and thus w € C*. However, w € C* contradicts the fact that u 
and v are generators of C*. For suppj(i;) C suppj(w) we get similarly a 
contradiction. rj 

Lemma 74. Letu,v,w € H and let A\ be a two dimensional cone generated 
by u and v, A 2 is a two dimensional cone generated by u and w and A 3 is a 
one dimensional cone generated by u. Let w = a\ -u+a2-v. Then A\ r\Ai = A 3 
if and only if a-i < 0. 

Proof. It is obvious that A3 C A\ n A^. Let x = v ■ u + w, where v > 
max(— ai,0). Then it is easily seen that x € A2 \ A3. It follows that x = 
(v + a\ ) ■ u + a-i ■ v and thus x € A\ n A2 if 02 > 0. So, A\ n A2 = A3 implies 
that a>2 < 0. Conversely, suppose that 

x' = X ■ u + X' ■ w e Ai n A 2 . 

Since x' € A 2 we have that A > and A' > 0. Then 

x' = X ■ u + X' ■ (ai ■ u + (12 ■ v) = ( A + A' • a\ ) ■ u + X' ■ (12 ■ v 

and since x' € Ai it follows that A' • 02 > 0. Thus if 02 < then A' = since 
A' > and a 2 ■ A' > 0. Hence x' € A3. □ 

Proof. (Proof of Theorem 75) We have that u£+ 1)! C* = H since U^+ 1)! Ci = 
W 1 . Therefore one of the following two cases holds 

1. There exists some two dimensional cone C* such that in the subspace H 
we have that x is in the internal of cone C* . 

2. There exist two dimensional cones C*,Cj such that in the subspace H 
we have that x is on the border of C* , x is on the border of Cj and x is 
in the internal of C* U C*. 

1 J 
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Suppose we have case 1 and let a and b be generators of the cone C*. 
Then a,b £ Ci and x = A ■ a + p, ■ b with A, \i > 0. Hence x £ d and 
supp^(x) = supp^a) Usupp^ft). Suppose that y $ C*. Then y = y\ -a + y^ -b 
with min(2/i,2/ 2 ) < and suppose without loss of generality that y\ < 0. 
According to Lemma 73 there exist k € supp^a) \ supp^ft). Then it follows 
that k $ supply), k € suppj(x). Thus supp^(x) is not a subset of supply), 
but this contradicts Corolary 77. Hence y £ C* and thus x,y £ Cj. According 
to Corollary 76 it follows that x <c y- 

Suppose we have case 2. Then there exist a,b £ H such that x and a are 
generators of cone C* and x and b are generators of cone C| . Let iti , tt 2 , v\ , v 2 
be such that y = u\ -x+m 2 -a = v\ -x+V2-b. If u\ < then it follows anagously 
to the proof in case 1 that supp^x) is not a subset of supply), which yields a 
contradiction again. Thus u\ > and analogously it follows that v\ > 0. We 
will prove that y e C*L)C*. Suppose that y g C*. Then it follows that it 2 < 0. 
Let a\ , ci2 be such that a = a\ ■ x + a 2 • b By Lemma 74 we have that a 2 < 0. 
Then y = U\ -x+it 2 - a = U\ -x+w 2 -(ai - x + a 2 -6) = (tti+tt 2 -ai) -x + (tt 2 -a 2 ) -6. 
Hence t> 2 = w 2 • a 2 > 0. Thus y e C* if 3/ $ C* and thus y £ C* U C*. Since 
x € C* <~)Cj it follows that x, y € Ci or x, y € Cj. Hence x <o J/ by Corollary 



13.2.1 Shift invariant counterparts 

We consider sequences (of nonnegative integers) of a given length T e N 
and given sum S € N. The set of such sequences is denoted by P(T,S). So, 
P(T,S) = {(xi,x 2 , . . . , xt) '■ Xi € %>>oVi,Yli=i x i = 3} an ^ this is asubmesh 
of dimension T — 1 of Z T . Let -D' be a multimodular matrix of size T x T 
induced by the submesh P(T, 5). 

Let x,x' £ P(T, 5). Then we say that x and x' are conjugate if they are 
cyclic permutations of each other. It holds that x and x' are conjugate if 
there exist finite (possibly empty) sequences v and w such that x = vw and 
x' = wv. It is easily seen that this conjugacy is an equivalence relation on 
P(T, S) and we write x ~ x' if x and x' are conjugate. We denote by x the 
conjugacy class of x € P(T, S) , which is the set of all cyclic permutations of 
x. By P(T,S) we denote the set of conjugacy classes of P(T,S). If x ~ x' 
then we also say that x and x' are shifts of each other. If x = y then we say 
that x is a representative of y. 

Let / : P(T,S) M be a function such that /(x) = f(x') if x ~ x'. 
Then we say that / is a shift invariant function. A shift mvariant function 
/ : P(T,S) -> S induces a function f ^P(T,S) -> S by /(») = /(x) where 
x e P(T,5) is a representative of 3/ e P(T,5). We denote by T shift (D') the 
set of functions mapping P(T, 5) to ffi which are induced by D' multimodular 
functions that are shift invariant. We use J 7shlft (D') to define a partial order 
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<mms on P(T, S) in the same way as r) was used to define the partial 
order < mm on Md- The partial order < mms is called the shift invariant 
multimodular order. 

Definition 79 Let a multimodular matrix D' induced by the submesh P(T, S) 
be given. Then for x,y £ P(T,S) we say that x < m ms y if f{x) < f(y) for 
every f £ T shiJt {D'). 

Let x € P(T,S). Then we say that x = (x\,X2, ■ ■ ■ ,xt) is regular if 
the induced infinite sequence x°° := {x\ , . . . , Xt, x\ , . . . , Xt, ■ ■ ■) is a bracket 
sequence and in that case the conjugacy class x € P(T,S) is also called 
regular. We denote by R(T, S) C P(T, S) the subset of regular sequences. 
The following lemma can be proved analogously to the proof for sequences 
of zeros and ones that is given in [74] . 

Lemma 75. LetT,S € N be given. Then there exists exactly one element in 
P(T,S) which is regular. 

We denote by lj = £5(T, S) the unique element of P(T,S) that is regular. 
Then we have the following theorem. 

Theorem 80 Let a multimodular matrix D' induced by the submesh P(T, S) 
be given and let f € T shlSt {I)'). Then a global minimum of f is attained in 
w(T,S). 

Corollary 81 We have that uj(T,S) is the smallest element for the partial 
order < mms on P(T,S). 

We have the multimodularj)rder < mm and we have the shift invariant 
multimodular order < mms on P(T,S). We also define a shift invariant cone 
order <c. 5 on P(T, S) which is the counterpart of the cone order <c- 

Definition 82 Let D' be as in Theorem 80. Then for u,v € P(T,S) we say 
that u <cs v if and only if there exist representatives u' ofu and v' of v such 
that u' <c v', where <c is the cone order for multimodular matrix D' and 
root r with r being a representative ofui(T,S). 

Note. According to Theorem 75 it holds that it' <c v' if and only if 
f(u') < f(v') for every / £ T(D',r). 

Corollary 83 Let D' be as in Theorem 80. Then for u,v € P(T,S) we have 
that U <mms v if u <c s v. 
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13.3 The graph order and the unbalance 

Let u = {u 1 ,u 2 ,...,u T ) G P(T,S) and for / = 0, 1, . . . , T - 1 let i*W := 
(ui + i,ui + 2, ■ ■ ■ , ut, 1*1,1*2, ■ ■ ■ , i*z) e P(T,S) be the Z-th cyclic permuta- 
tion of u. For a sequence u £ P{T, S) we define a counting function 
k u : {0,1,..., T} — ► Z by K u (n) = J2t=i Ut an( ^ we define a discrepancy 
function <j> u : {0, 1, . . . , T} — ► Q by ^(n) = k„(tj) — n • ^ for n = 0, 1, . . . , T. 

Lemma 76. For u e P(T,S) and I e {0,1,. ..,T - 1} we have that 
<t> u w( n ) — f or n = 0,1,. . . ,T if and only if <f> u (l) = min ra= o i i i ... i T <f>u(n) and 
<fi u m(n) < for n = 0, 1, ... , T if and only if <f> u (l) = max„ =0 ,i,...,T 4>u(n). 

Proof. Suppose that <j> u o) (n) < for some n € {0, 1, . . . , T— 1}. Then <f> u ((l + 
n) (mod T)) = <j> u {l) +^„(o(n) < ^(/). Thus ^(/) = min„ =0 ,i,...,T ^«(n) 
implies that (f> u o)(n) > for n = 0,1,..., T. Suppose that <f> u (l') < 
4> U {1) for some / ^ /' e {0, 1, . . . , T - 1}. Then ^ (0 ((/' - /) (mod T)) = 
^(/') - <t>u{l) < 0. Thus 4> u m{n) > for tj = 0, 1, . . . ,T implies that 
^t(Z) = min„ = o,i,...,T <fiu(n). Analogously it follows that (f> u o)(n) < for 
n = 0, 1, . . . , T if and only if ^(Z) = max ra= o l i l ... l T <fiu(n). rj 

While the multimodular order < mm and the cone order <c are partial 
orders on Z T we define next a partial order on P(T,S). This induces partial 
orders on P(T, S) from which we derive our bounds for the expected average 
waiting time. 

Definition 84 For u,v € P(T,S) we say that u < v if K u (n) < K v (n) for 
n = l,2,...,T. 

We have the following (see [74] where it is proved for sequences of zeros 
and ones). 

Proposition 20. Let u e R(T,S). Then u' e R(T,S) if and only if u~u'. 
Moreover the partial order < on P(T,S) induces a total order on R(T,S). 

Since R(T, S) is finite it follows from Proposition 20 that R(T, S) contains 
a greatest element for this order. We denote this greatest element by ui{T, S) 
or just ui if no confusion is possible. For this element w the partial sums 
K w {n) for n = 1, 2, . . . , T are as great as possible under the restriction that uo 
is regular. For example w(7,4) is the sequence (1, 1,0, 1,0, 1,0). From [74] we 
have the following lemma which can be used to determine uo{T,S) quickly. 

Lemma 77. 



K»(T,s)(n) =\n- -] for n = 0,1,..., T. 
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We have seen in Proposition 20 that the regular sequences R(T, S) form 
a conjugacy class in P(T, S). It follows that u £ R(T, S) if and only if u is 
a representative of u)(T, S) £ P(T, S). Combining Lemma 76, Proposition 20 
and Lemma 77 we obtain the following theorem in which the partial order 
< is used to give a characterising property of the conjugacy class R(T, k) = 
w(T, S) of regular sequences in P(T, S). 

Theorem 85 Every conjugacy class u of P(T, S) contains an upper bound 
of R(T,S), i.e for every u £ P(T,S) there exists a v £ P(T, 5) such that 
v ~ u and v >z w for every w £ R(T, S). 

We have the following preorders < g , < g and < g on P(T,S) called the 
primal or upper graph order, the dual or lower graph order and the strong 
graph order respectively. 

Definition 86 Letu,v £ P(T,S). Thenu< g v if there exist u',v' £ P{T,S) 
such that u' £ u, v' £ v and < <f> u i (n) < 4> v * (n) for n = 0, 1, . . . , T and 
u < g v if there exist u",v" £ P(T,S) such that u" £ u, v" £ v and > 
4>u" {n) > 4> v " {n) for n = 0, 1, . . . , T. Further u< g v if u<gV and u < g v. 

The preorders <j, < g and < g on P(T, S) are not partial orders. Namely, 
if it, v £ P(T, S) are cyclic permutations of each other then it is easily seen 
that u < g v and v < g u. Thus these orders are not antisymmetric. However, 
they induce (see Definition 87 ) partial orders on P(T,S). 

Definition 87 Let u,v £ P(T,S) and u' £ P(T,S) be a representative of u 
and v' £ P(T,S) be a representative of v. Then we say that u < g v if and 
only if u' < g v' and similar for the orders <j and < g . 

In [74] it is shown that these induced graph orders on P(T, S) are partial 
orders. 

For a sequence u £ P(T, S) we have a primal unbalance I(u) and a dual 
unbalance I_{u). 

Definition 88 Let u £ P(T,S). Then the primal unbalance of u is 

1 T S 

where u' £ u such that cf> u < (n) > for n = 1, 2, . . . , T. The dual unbalance 
of u is 

1 T S 

Tl=l 

where u' G u such that 4> u > (n) < for n= 1, 2, . . . , T. 
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Remark 29. For it £ P(T, S) the primal unbalance is well denned. Namely, 
by Lemma 76 there exist it' £ u such that <f> u i (n) > for n = 1, 2, . . . , T. 
Moreover, if it', it" £ u are such that <j> u '{n) > and (f> u "{n) > for n = 
1, 2, . . . , T then En=i ( n ) = El=i ( n ) ( see Theorem 89). Analogously 
it follows that the dual unbalance is well defined. It is easily seen that for 
every sequence it £ P(T,S) it holds that J(it) > and J(it) > 0. Namely, 
for it' £ P{T, S) we have that <f> u > (n) > if and only if k u * (n) > \n ■ ^] and 
<t>u> (n) < if and only if k u > (n) < [n ■ . Moreover, a sequence it € P(T, S) 
is regular if and only if /(it) = if and only if /(it) = (see [74]). 

Theorem 89 Let u £ P(T,S). Then the following statements are equivalent 
fori G {0,1,.. .,T-1}. 

(i) J(it) = ^ • ELi (n) - \n ■ f 1). 

ELi K«o(n) = max i= o,i,..., T -i ELi 

(iii) min„ =0 ,i,...,T-i <j> u m{n) = <fi u m(Q) = 0. 

(iv) 4> U {1) = min i=0 ,i,...,T-i 4>u{i)- 

Proof. By Lemma 76 we have that (iii) implies (iv) and vice versa. We now 
prove that (ii) implies (iii). Suppose there exists some t £ {1, 2, . . . , T — 1} 
such that <j) u o){t) = —fi with \x > 0. Let x be the prefix of length t of 
tt^ and let y be the suffix of length T — t of it^. Then it^ = xy and let 
z — yx = it« i+t ) ( m ° dT )) g u. Then ^„( (n) = <j> z (n - t) - n for n = 
t, t + 1, . . . ,T and (f> u o)(n) = 4> z {n + T — t) — fi for n = 1, 2, . . . ,t — 1. Hence 

eLi («) = eLi - T • m and thus 

XT T T 

y~] K uw( n ) = y~\ K z(n) - t ■ p. < y~] K z( n ) ^ max XI K ua)( n ), 

7=0,1, ...,T—1 

71=1 71=1 71 = 1 71=1 

which contradicts (ii). From the definition of the primal unbalance it follows 
directly that (iii) implies (i) and to finish the proof we show that (i) implies 

(ii) . Suppose that En=i K «<o(rt) < max i=0 ,i,...,T-i El=i K «<o(rt) and let / ^ 
/' e {0,1,..., T- 1} such that 

El=i K 7 l (i')(«) = max i=0 ,i,...,T-i En=i K «(o(«)- Since we have proved that 
(ii) implies (i) it follows that 

1 S 1 _ S 

= - ■ ^2(K u v>)(n) - \n ■ -1) > - ■ ^(Mo(n) - fa ■ ^1), 

71 = 1 71=1 

which contradicts (i). rj 

The following result follows immediately from the definitions. 

Lemma 78. Let u,v £ P(T,S). If it u then J(it) < /(f). //it -< g v then 
I(u) < I(v). Ifu< g v then I(u) < I(v) and I(u) < I(v). 
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We have the following theorem, which shows that the primal and dual 
unbalance are multimodular with respect to the L - Triangulation (see Section 
12.2). 

Theorem 90 The primal unbalance and the dual unbalance are shift invari- 
ant and multimodular on P(T, S) with respect to the base d\ , di , . . . , dr where 
di = &i — ej+i for i = 1, 2, . . . , T — 1, dr = ex — &i and e; is the i -th unit 
vector of length T for i = 1, 2, . . . , T. 

Proof For u € P(T, S) let f(u) = I(u) and g(u) = I(u). Thus / : P(T, S) -► 
Q is the primal unbalance function and g : P(T, S) — > Q is the dual unbalance 
function. From the definition it follows that if u',u" € P(T,S) are cyclic 
permutations of each other then /(«') = f{u") and g(u') = g(u"). Thus 
the primal unbalance and dual unbalance are shift invariant. To prove the 
multimodularity of the primal unbalance function / we have to show that for 
every u e P(T,S) and i,j e {1,2,...,T}, i ^ j it holds that 

f(u + di) + f(u + dj) > f(u) + f(u + di + dj). (13.3) 

We first show that for every x e P(T, S) and i e {1, 2, . . . , T} it holds that 

f{x + di)<f{x) + ± (13.4) 

Namely, put y = x + di. Without loss of generality we can assume that 
ELi K v( n ) = max i=0 ,i,...,T-i ELi K yW {n). Then f(y) = t-ELi( k j/( k )- 
\n ■ §;] ) and 



fix) = f(y - dk) 

1 T S 



>^-(E(»»-r«-^i)-i) 



= f(y) - if, 

by Theorem 89. So, (13.4) holds and next we show that (13.3) holds. Without 
loss of generality we assume that min Tl=0i i i ... i T-i (f>u{n) = <Au(0) = 0. Then 
by Theorem 89. Suppose that i T and j ^ T. Then it is easily seen 
that K u+di {n) > K u (n), K u+dj (n) > K u (n) and Ku+di+dj (n) > K u (n) for 
n = 0, 1, . . . , T. Hence 

min d> u+di (n)= min 6 u+d {n) 

n=U,l,...,l —1 n=U,l,...,l —1 

= min 4> u+di+d An) 

n=0,l,...,T-l i< 

= min 4> u (n) 

n=0,l,...,T-l 

= 0. 
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From the definition of the primal unbalance it is easily seen that /(it + 
di) = /(it + dj) = /(it) + ^ a nd f(u + di + dj) = /(it) + ^ and thus 
f(u + di) + f(u + dj) = /(it) + /(it + di + dj). It remains to show that (13.3) 
holds in case i = T or j = T and we can assume that j =T and i^T. Then 
from the foregoing we have that /(it + di) = /(it) + ^. Moreover by (13.4) 
we have that /(it + di + dj) < /(it + dj) + ^. Hence 

f(u + di) + /(it + dj) > (/(it) + 1) + (/(it + d z + dj) - 1) 
= f(u)+ f(u + di + dj). 

Thus the primal unbalance function / is multimodular and it follows analo- 
gously that the dual unbalance function g is multimodular. rj 

Note. Since the primal and dual unbalance functions are shift invariant 
on P(T, S) they induce functions on P(T, S) by I(v) = I(u) and I_(v) = J(it) 
if it e P(T, S) is a representative of v € P(T, S). Then by Theorem 90 we have 
for these induced primal and dual unbalance functions that they are element 
of J rs ' l ^*(i?), where D is the multimodular matrix having the multimodular 
base di , d 2 , . . . , d T as row vectors. 

Corollary 91 For this multimodular base d\ , d 2 , . . . , rfr we have for u,v € 
P(T,S) that it <mms v implies that I(u) < I(v) and I_{u) < I_{v). 

13.4 Relations and counterexamples 

13.4.1 The shift invariant cone order does not imply the graph 
order 

Consider the following example. Let it = (1,3,2,2,3) € 7^(5, 11) and 
v = (2,1,3,2,3) e V(5,ll). Thus we consider sequences of length 5 and 
sum 11. It is easily seen that it and v are not ordered for the dual lower 
graph order (but they are ordered for the primal upper graph order). So, 
it and v are not graph ordered. However, we now first show that u and v 
are ordered for shift invariant multimodular functions with respect to the 
standard base do,di,d2,d 3 ,d 4 where di = — e^+i for i = 1,2,3,4 and 
do = (—1,0,0,0,1). Namely, for a shift invariant multimodular function / 
we have that /(it + d 4 ) = /((l, 3, 2, 3, 2)) = f(y) and also /(it + d x ) = 
/((2, 2, 2, 2, 3) = /((2, 2, 2, 3, 2)) = f(u + d 1 +d 4 ) by shift invariance. By mul- 
timodularity we have that /(it + d\) + /(it + d 4 ) > /(it) + /(it + d\ + d 4 ) and 
thus /(it) < f(u + d 4 ) = /(d) for every multimodular shift invariant function 
/■ 

Conclusion. From this it follows that the shift invariant mutimodular 
order does not imply the graph order and thus the shift invariant multimod- 
ular order does not imply the shift invariant cone order or the shift invariant 
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cone order does not imply the graph order (or both do not hold). 

We now investigate whether it <c s v. To do this we take the regular 
sequence r = (3,2,2,2,2) as root of all the cones we consider and we let 
u',v' run through all the shifts of it and v respectively. Then we check 
whether it' < v' for some cone. Doing this we find that it and v are in- 
deed ordered for the shift invariant cone order. Namely, for it' = (3, 1,3,2,2) 
and v' = (2,1,3,2,3) we have that it' = r + do + d\ + d% + d* and 
v' = r + 2do + d\ + d$ + d±. Hence, if we consider the cone generated by 
b\ = do , 62 = do + d\ , 63 = do + d\ + d$ and 64 = do + d\ + d$ + d± then 
it' = r + 64 and v' = r + b\ + 64 . By the way, this u' and v' is the only pair 
of shifts that are cone ordered for some cone with root in r. Since u' <c v' 
and thus it <c s v we have the following conclusion. 

Conclusion. The shift invariant cone order does not imply the graph 
order. 

Remarks. The explanation for the fact that the ordering of it' and v' in 
this cone does not imply the lower graph order is the following. For it' you have 
to start with the second coordinate to get the graph for the lower graph order 
and for v' you have to start with the first coordinate. This is possible, because 
for r you have the start at the second coordinate for the lower graph order, 
while the generating base vectors of this cone " suggest" the lower graph order 
for starting at the first coordinate. One of the consequences is that this cone 
does contain other regular sequences than r. Namely, r + b\ = (2, 2, 2, 2, 3) is 
for example another regular sequence in this cone. It turns out that the cone 
order in such cones does not imply the graph order. 

If you consider the mirrored sequences of it and v then you get a similar 
problem with the upper graph order instead of the lower graph order. 
Note that it and v are ordered for the unbalance (despite that they are not 
ordered for the graph order). Namely, for the primal (upper) unbalance I 
we have that /(it) = 1 < 2 = I(v) and for the dual (lower) unbalance J we 
have that J(it) = 1 = J(d). Of course this order for the unbalance follows 
immediately from u < mms v and Corollary 91. 

13.4.2 The shift invariant multimodular order does not imply the 
shift invariant cone order 

A counterexample is given here to show that the shift invariant multimodular 
order does not imply the shift invariant cone order. In fact it is shown that 
the shift invariant cone order is not even a partial order, because it is not 
transitive. 

Counterexample. Again we consider sequences in P(5,ll) and we 
have the same multimodular standard base {di}i=o,i...,5 as before. Let it = 
(1, 2,3, 2,3), v = (2, 2,3, 3, 1) and w = (2, 1,0,4,4) and as regular sequence 
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we have again r = (3, 2, 2, 2, 2). Now we have for the shift invariant cone or- 
der that u <cs v and v <c s w. However, we do not have that u <c s w. 
Namely, u' = (3,2,3,1,2) = r + d 3 = r + h and v' = (2,3,3,1,2) = 
r+d 2 + 2d 3 +d 4 + d = r+61+64, where bi = d 3 , b 2 = d 3 +d , b 3 = d 3 +d +d 2 
and 64 = d 3 + do + d 2 + d 4 . Hence u <c s v. 

Further v" = (3,1,2,2,3) = r + d + d x = r + b 2 and w" = (2,1,0,4,4) = 
r + 4d + 3d\ + 2d 2 + 2d 4 = r + h + b 2 + 264, where 61 = do, b 2 = d + d\, 
b 3 = do + d\ + d 2 and 64 = do + d\ + d 2 + d 4 . Hence v <c s w. However, there 
exist no cone with root in r such that some shift of u is smaller in that cone 
than some shift of w. Hence, u and w are not shift invariant cone ordered, 
while from u <c s v and v <o s w it follows that u and w are ordered for the 
shift invariant multimodular order. 

Remark 30. So, it turns out that the shift invariant cone order is not a par- 
tial order, because it is not transitive. It is natural to consider the smallest 
preorder which contains the shift invariant cone order and is transitive. It is 
easily seen that the shift invariant multimodular order implies this order, but 
we do not know whether the converse is also true in which case this order is 
equivalent to the shift invariant multimodular order. 

13.4.3 The graph order does not imply the shift invariant 
multimodular order 

In this counterexample we show that the graph order does not imply the shift 
invariant multimodular order. 
Consider the sequences 

u = (0,0,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,1,1,0,0,1,1,1,1,1,0,1,0,1,0,1) e 

P(32,15) and 

v = (0,0,0,0,0,0,0,0,0,0,0,1,1,1,0,1,1,0,1,1,0,1,1,0,1,1,0,1,1,1,0,1) e 

P(32, 15). Then we have that u < g v. Thus u and v are ordered for the graph 
order and we have that /(it) < I(v) and I_{u) < I_( v )- However, it does 
not hold that f(u) < f(v) for every shift invariant multimodular function 
/ : P(32, 15) — ► K. Namely, we consider u and v as period cycles for the split- 
ting sequences corresponding to the admission of arriving customers to some 
server. Suppose that the interarrival times are deterministic and equal to 1 
and the service times are deterministic and equal to (1 +e), where e is a small 
positive number. For x € P(32, 15) let W(x) be the average waiting time of 
customers admitted according to x. The average waiting time is multimodu- 
lar with respect to the standard base of the L - Triangulation ( see Chapter 
12) and since the traffic intensity for the server is smaller than one we have 
that the average waiting time function W is also shift invariant (see [74]). 
Thus W is a shift invariant multimodular function. However, for the given 
interarrival and service times we have that W(u) = j| ■ e and W(v) = j| ■ e. 
Thus W(v) < W(u) for these service times and thus u and v are not ordered 
for the shift invariant multimodular order. 
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13.5 Bounding the Difference in Waiting Times for One 
Queue 

In this section we give a bound for the expected average waiting time for 
customers routed to a single server queue according to a routing sequence of 
zeros and ones. This bound depends on the unbalance of the routing sequence. 
To state the results we first have some definitions and notations. 

Let it and v be infinite periodic integer sequences. Then we say that it 
is equivalent to v, it ~ v if there exists a finite sequence w such that w is a 
period cycle of both it and v. 

In our application of routing sequences we have in general that equivalent 
infinite periodic sequences have the same performance. Let V be the set of in- 
finite periodic sequences of zeros and ones (modulo the equivalence relation) , 
1Z CP the subset of regular periodic sequences and for d £ Q, < d < 1 let 
V(d) C V be the subset of sequences with density d. We denote the regular 
sequence in V(d) by u(d) or just u>. Note that if it' £ V(T, k) is a period cycle 
of it £ V then it € V{d) where d = ^. We define the unbalance for infinite 
periodic sequences of zeros and ones as follows. 

Definition 92 Let u £ V and let it' € V(T, k) be a period cycle of it. Then 
we define the (primal) unbalance of u as /(it) := I{u'). 

The unbalance is well denned on V. Namely, if it' and it" are both period 
cycles of it then J(ii') = J(it"). 

Let {Ti}i = i t 2,... be a sequence of arrival times of customers, with the 
convention that T x = 0. Put 6* := T i+1 - T { for i = 1,2,.... Then {<5J 
is the sequence of interarrival times. Further these arriving customers are 
routed to a server according to some routing sequence it = (iti, it2, . . .) of 
zeros and ones. For such routing sequences we have the counting function 
Ku(n) = E"=i u t and we define a partial order < that is similar as the order 
on P(T,S). Namely, we say that u ■< v if K u (n) < n v (n) for n = 0,1, — 
Further we define the the following related function v u {i) : Z> — ► Z with 
v u {j) = mm{n G Z> : K u (n) = j} and we put T u (j) = Ei=m]^(t„(j_i),i) & 
for j = 1,2, — Then r u (j) is the time elapsed between the routing of the 
(j — l)-th and j-th customer to the server according to routing sequence 
it. If we put X u (j) = J2i=i T u{k) for j = 1,2,... then X u (j) is the time 
at which the j — th customer is routed to the server according to routing 
sequence it. We have a sequence of service times {<Tj}j=i,2,..., where aj is the 
service time of the j-th customer that is routed to the server according to the 
routing sequence. Further we define W u (j) to be the workload for the server 
at the moment the j-th customer is routed to the server according to routing 
sequence it. In other words W u (j) is the waiting time for the j -th customer 
that is routed to the server. We assume that the server starts empty at Ti = 
and thus W^l) = 0. If the interarrival times {6i} and service times {aj} are 
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random variables then we say that W(u) is the almost sure long-run average 
waiting time of customers routed to the server according to routing sequence 
u if linim^oo ^ • Y^jLi = W(u) with probability one. From ergodic 

theory we have the following theorem. See [74] for a proof of this theorem. 

Theorem 93 Suppose that the interarrival times {<5;} of customers arriv- 
ing at the system are independent and identically distributed (i.i.d.) random 
variables with mean 6 and the service times {aj} of the considered server 
are i.i.d. random variables with mean a and independent of the interarrival 
times. Further let u' and u" be routing sequences of zeros and ones that are 
both representatives of some u £ V(d) where d £ Q (thus they have a com- 
mon period cycle) and j-d < 1. Then W(u') and W(u") exist and are finite. 
Moreover W(u') = W(u"). 

Let 6 , a and d be as in Theorem 93. Then we say that p := j ■ d is 
the traffic intensity for the server. By Theorem 93 all the routing sequences 
which are representative of some u £ V(d) have the same long-run average 
waiting time if the stability condition p < 1 is fulfilled. Therefore in this case 
we denote this long-run average waiting time simply by W(u). The following 
theorem which is obtained by a sample path argument is the main result of 
this section. 

Theorem 94 Let the interarrival times {Si} and the service times {aj} be 
as in Theorem 93. Further let u € V(d) for some d £ Q such that p < 1 and 
let u> = u>(d) e V(d) be the regular sequence of density d. Then 

W{u) < W{oj) + - ■ I(u). (13.5) 

Remark 31. From the results of Chapters 2 and 4 we have that W(u) > 
W{lo). 

We give an outline of the proof of Theorem 94. For a complete proof for 
this and following results in Sections 13.4 and 13.5, see [74]. We first assume 
that the interarrival times {<5i}i=i,2,... and service times {o'j}j=i,2,... are fixed 
sequences of non- negative real numbers. Then we have the following lemma. 

Lemma 79. Let u = (ui, 1*2, . . .) and v = (vi, V2, ■ ■ ■) be routing sequences 
of zeros and ones and suppose that u<v. Then 

W v (j) + X v (j) < WJJ) + X u (j) for every j e N. (13.6) 

Note that W u (j) + X u {j) is the moment that the server starts serving 
the j-th customer that is routed to the server according to u. So, Lemma 79 
states that if u ■< v then the moment when the service of the j -th customer 
that is routed to the server starts, is for v not later than for u. This can be 
proved by induction to j. 
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For every j £ N we have that X u (j) — ^v(j) = Yli=j\j) This is a sum of 
interarrival times <5;, where the number of terms in the sum is v u {j) — v v (j). 
Therefore, if we put N uv (m) = Y^jLii^uU) — v v {j)) for m = 1, 2, . . . then we 
have the following Corollary by Lemma 79. 

Corollary 95 Let it and v be routing sequences with u < v. For every m £ N 
we have that 

7X1 7X1 7X1 Vu(j) — 1 

E^0')^E^') + E E ^. (i3.7) 

i=i i=i i=i i=v v (j) 

In the double sum the number of terms is N uv (m). 

It can be shown that if u = (u')°° and v = (v')°° for some u', v' € V(T, k) 
and uj(T, k)<u' < v' then 

m^oo m k 

Let w € ^(rf) be as in Theorem 94. Then by Theorem 85 there exist T, k £ 
N with d = j; and it" € ^(T, fc) such that it' := (it")°° is a representative of 
it and u' < it' where u' = ui(T, fc)°° is a representative of ui(d). By Theorem 

93 we have that both W(u) = W(u') and W{u) = W(w') exist. We make a 
coupling between the interarrival times for it' and ui' and we also do this for 
the service times. After the coupling we can apply Corollary 95 and (13.8) 
and it can be shown that 

W(u') - W{J) < 6 -f ■ (I(u) - IM) = 8 - ■ J(it), 

which proves Theorem 94. 

Remark 32. From this proof it follows that it is possible to refine Theorem 94 
as follows: Let u,v € V(d) be as in Theorem 94 and let oj' be a representative 
of u(d) as above. Then, if there exist representatives it' and v' of it and 
v respectively such that ui' < u' < v' (The reader can check that this is 
equivalent to the following: there exist T, k £ N and period cycles it" and v" 
of it and v respectively such that d = |r , u",v" £ V{T, k) and it" <j v") it 
holds that 

W{v) - W{u) < -(/(«) - T(it)). (13.9) 

The condition p < 1 in Theorem 94 is just necessary to apply Theorem 93. 
However, if both the interarrival times and the service times are deterministic 
then Theorem 93 also holds if p = 1 (see [112]). Thus in that case Theorem 

94 also holds if p = 1 and in fact we have for that case the following stronger 
result from which it follows that the bound of Theorem 94 is tight. 
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Theorem 96 Suppose that the interarrival times and service times are de- 
terministic and equal to 6 respectively a. Let u £ V(d) for some d e Q such 
that 6 = da, hence p = 1, and let u> = u(d) be the regular sequence of density 
d. Then 

W(u) = W{oj) + - ■ I(u). (13.10) 

Outline of the proof. Let u>' be a representative of u(d) as above and 
let u' be a representative of u such that u' <u'. Then it can be shown that 
W u > (j) + \ u ' (j) = Wu> (j) + {j) = {j - for every j £ N. Hence we have 
for every meN that (13.7) holds with equality for it' and u>' with 6, = 6 for 
i = 1, 2, — From this it follows that (13.5) also holds with equality. 

Remark 33. The upper bound can be generalized to a sequence of multimod- 
ular functions satisfying the relations of Section 1.3. Indeed, it follows from 
the results in the Section 1.3 that the regular or bracket sequence is the min- 
imal admission sequence. If the density is rational then the corresponding 
regular admission sequence is periodic. In order to obtain upper bounds as 
in Theorem 94 and Theorem 99 we need a generalization of Lemma 79, es- 
pecially the relation (13.6). It is easily seen that for multimodular functions 
fk corresponding to W u (j) it suffices that 

fk(ai, ...,ai + l,aj+i - 1, . . . , a k ) < fk(ai, ■ ■ ■ ,ai,ai+i, . . .,a k ) + 6, 

for 1 < / < k — 1 and some constant 6. Clearly, also other multimodular 
models treated in this monograph do satisfy this relation e.g. the models of 
Chapter 9 . 

13.6 Routing to Parallel Queues 

In this section we derive upper bounds for the expected average waiting 
time for routing arriving customers to N > 2 parallel queues according 
to some periodic routing policy U = (Ui, U2, ■ ■ ■)■ The routing sequence 
U = (Ui,U2, ■ ■ •) can be seen as an N - word on the alphabet {1,2,..., N}. 
Further to every letter i 6 {1,2,..., N} corresponds a sequence of zeros and 
ones u 1 = (u\, u 2 , . . .) via the support of the letter, i.e. u\ = 1 if and only if 
U n = i. We call u 1 the routing or splitting sequence for queue i (see Chapter 
6). Extending the notion of unbalance for periodic sequences of zeros and 
ones we define a (total) unbalance 0(U) for periodic integer sequences and 
corresponding words on some finite alphabet. Then we extend the result of 
Theorem 94 for one queue to multiple queues by using the unbalance 0(U) 
of U. 

Let di, . . . , d N e Q >0 with J^iLi d% = 1- We denote by Q(di, d 2 , . . . , d N ) 
all the infinite periodic words U on the alphabet {1,2,..., N} for which u % € 
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V{di) for i = 1, 2, . . . , N. Further for T, Jfci, k 2 , . . . , k N e N with Y,i=i k i = 
T we denote by 2({T}, fci, fc 2 , • • • , fc/v) all the N-words on the alphabet 
{1,2,..., N} for which every subword of length T contains exactly ki letters 
i for i = 1,2,..., N. Note that Q({T}, k u k 2 , . . . , k N ) C Q(^, ^, . . . , fy). 
For example if U = (1, 2, 1, 2,1,3, 1, 2, 1, 1, 2, 3)°° then U £ Q({12}, 6, 4, 2) 
C Further w 1 = (1,0,1,0,1,0,1,0,1,1,0,0)°° e P(§), u 2 = 

(0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0)°° e V{\) and w 3 = (0, 0, 0, 0, 0, 1)°° e V{\). 

Let ip be a routing policy and £/ the corresponding routing sequence. Then 
for t € N we define W(t) = W^(t) as the waiting time of the t -th arriving 
customer if policy ip is applied, which is the remaining workload for server 
Ut at the moment that the t-th customer arrives. We assume that all the 
servers i £ {1,2, ... , N} are empty at I\ = and thus W(t) = if t = v u i (1) 
for some i € {1,2,..., N}. If the interarrival times and service times of the 
various servers are random variables then we say that W(ip) = W(U) is the 
almost sure long-run average waiting time of the arriving customers routed 
according to policy ip if lim^oo ^-J2t=i W^(t) = W(ip) with probability one. 
Let {Si} be the sequence of interarrival times and let {<t]} be the sequence 
of service times of server i for i = 1, 2, . . . , N, i.e a] is the service time of the 

j - th customer that is routed to server i. We define W (ip) = W (V) as the 
almost sure long-run average waiting time of customers routed to server i if 
policy ip is applied in the same way as we did in the previous section. The 
only differences are that routing sequence u is replaced by routing sequence 
u l and the sequence of service times {cr,} is replaced with the sequence of 
service times {fj}. From Theorem 93 we obtain the following theorem. 

Theorem 97 Suppose that the interarrival times {6i} of customers arriving 
at the system are i.i.d. random variables with mean 6 and for every i € 
{1,2, ... ,N} the service times {cr]} are i.i.d random variables with mean 
en, and independent of the interarrival times. Let ip be a routing policy that 
corresponds to some word U € 2(^1,^2, • • • , ^jv) such that pi := ^--di < 1 for 
i = 1,2, . . . ,N . Then W(ip) exists and is finite. Moreover W(ip) = Y^i=i ^ " 
W{ip). 

Definition 98 Let U £ Q(d\, d 2 , . . . ,d N ) for some di e Q >0 with J2iLi d% = 
1. Then we define the (total) unbalance ofU as 

N 

0{U):=YJW). (13.11) 

i=i 

If it is clear from the context what is meant we just say unbalance and not 
total unbalance. Let the interarrival times {Si} and the service times {<rj} for 
i = 1, 2, . . . , N be as in Theorem 97. Further let di £ Q >0 for i = 1, 2, . . . , N 
with YliLi di = 1. Recall that ui(di) is the regular sequence of density di. If 
W (ui(di)) exists and is finite for i = 1,2, ... ,N then we put 
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H f N — 

R = R(d x , d 2 , ...,d N ) d i ■ w \"( d i))- ( 13 - 12 ) 

i=i 

Remark 34- If the interarrival times and service times are random variables 
then for the existence of R it suffices that pi < 1 for i = 1,2, ... ,N, while 
if the interarrival times and service times are deterministic then it suffices 
that pi < 1 for i = 1,2, ... ,N. It is possible to extend the definition of R 
and some of the results to the case of irrational di. In general R depends on 
the distribution of the interarrival times and service times and in some cases 
it is possible to compute R explicitly. See Chapter 8 and [112] for an exact 
computation in case of deterministic interarrival and service times and see 
[101] for computations and bounds in general. 

Combining Theorem 94 and Theorem 97 we obtain the following theorem. 

Theorem 99 Let the interarrival times {6i} and the service times {<rj} for 
i = 1, 2, . . . , N be as in Theorem 97. Let di £ Q>o for i = 1,2,.. .,N with 
J2iLidi = 1- Suppose that a routing policy ip is applied with corresponding 
word U e Q(di, cfe, ■ ■ ■ , rfjv) and pi < 1 for i = 1,2,..., iV. Then 

W(ip) - R<6 -0{U). 

From Theorem 26 we have that W(ip) > R, which is the lower bound 
obtained by replacing the routing sequence to queue i by the regular routing 
sequence with the same density for any queue i. Hence we have the following 
bounds. 

R< W(tp) <R + 6-0(U). (13.13) 

Remark 35. Suppose that we have a queueing system where the arrivals are 
according to a Poisson process with parameter A. Suppose that the service 
times of server i are exponentially distributed with parameter pi. If di, the 
fraction of jobs that is routed to server i, equals ^- for some qi £ N, then 

W (iv(di)) can be calculated in the following way. For the routing sequence 
ui(di) we have that among every qi arriving jobs exactly one job is routed to 
server i. So, the interarrival times at server i consist of qi Poisson arrivals with 
parameter A. Hence the interarrival times for the queue of server i are Erlang 
distributed, namely according to an Ef distribution. Thus W (ui(di)) is the 
same as the average waiting time for a Ef /M/l queue, where the parameter 
of the service times is pi. So (see [56]) if Xi € (0,1) is a solution of the 
equation 

x=(- )*, (13.14) 

A + pi - pi ■ x 

then 

^M*)) = - n' v ( 13 - 15 ) 

Pi ' \± Xi) 
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In the following example we have calculated W (w(di)) for i = 1,2,3,4 
by applying (13.15). Further we have explicitly calculated the lower bound 
R and upper bound R + 6 ■ 0(U) of (13.13) for W(ip), where U is the word 
corresponding to the applied routing policy ip. 

Example 10. We consider a queueing system with 4 parallel servers where 
the arrivals are according to a Poisson process with parameter A = 11. Hence 
for the mean interarrival time 6 we have that 6 = jj . The arriving jobs are 
routed to the servers according to the policy ip that corresponds to the word 

= (1,2, 3,1,4, 2, 1,3,1,2, 4,3)°°e <2({12},4, 3, 3, 2) C Q(I, I, I, I). 

Then we have that /(w 1 ) = ^, I{u 2 ) = 0, I{u 3 ) = ±, I{u 4 ) = and 
thus 0(U) = |. For every i € {1,2,3,4} the service times are exponentially 
distributed with parameter ^ and Hi = 4, fi 2 = fJ>3 = 3 and 114 = 2. Then 
we find by (13.15) that W 1 ^^)) = 1.7792 (rounded to 4 decimals) and 
thus by Corollary 94 we have that Vf 1 ^ 1 ) < 1.7792 + 3 • ^ • ^ = 1.8019. 
Similarly we have that W 2 {oj{\)) = W 2 {u 2 ) = 2.2105, W 3 {oj{\)) = 2.2105, 

W 3 (u 3 ) < 2.2408 and W^i)) = W 4 ^ 4 ) = 3.0732. Hence by (13.12) we 
have that 

R = l-W 1 ^ 1 -)) + \ -W\u(\)) + \-W\u{\)) + I ■ W 4 ^ 1 -)) = 2.2105 

and R + 6 ■ 0{U) = 2.2105 + ^ . I = 2.2257. So, by (13.13) we have that 
2.2105 < W(tp) < 2.2257. 

The following theorem provides conditions under which the right-hand 
side inequality of (13.13) actually holds with equality. 

Theorem 100 Let the interarrival times {^} be deterministic equal to 6 
and the service times {<rj} be deterministic equal to Oi fori = 1,2, . . . , N . 

Let di € Q>o for i = 1, 2, . . . , N with J2iLi d, = l. Let pi, qi € N be such that 
di = with gcd(pi, qi) = 1 for i = 1, 2, . . . , N. Suppose that a routing policy 
ip is applied with corresponding word U € Q(di, d 2 , ■ ■ ■ , rfjv) and pi = 1 for 
i = l,2,...,N. Then 

W(iP) = R + 6-0(U) = 6- ^I-^-L + ([/)j . 

By combining Theorem 96 and Theorem 97 we get the first equality of 
Theorem 100. See [112] for the second equality which follows from the com- 
putation of R for this case. 
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Example 11. We consider a queueing system with 3 parallel servers where 
the interarrival times are deterministic and equal to 6 = 3. The arriving jobs 
are routed to the servers according to the policy ip that corresponds to the 
word 

= (1,2,1,2,1,3,1,2,1,3) 00 G Q({10},5,3,2)C Q(i A I). 

All the service times are deterministic and for server 1 they are equal to 
a i = 6, for server 2 equal to 02 = 10 and for server 3 equal to 03 = 15. Hence 
Pi = 1 for i = 1,2,3. Further /(it 1 ) = 0, I(u 2 ) = I(u 3 ) = ± and thus 
0(U) = |. So, according to Theorem 100 we have that 

v ^ V 2 4 20 10 5 J 10 
which can be checked by direct calculation. 
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14.1 Introduction 

As mentioned before, the control of a stream of incoming customers into 
several queues in parallel is a difficult problem. When the number of queues 
is not larger than two, then the problem has been solved in several cases. In 
the case with full information on the state, the optimal policy has switching 
curves and can be computed using dynamic programming (see [58]). When 
the system is controled when no information is available, the optimal policy 
is a Sturmian sequence as shown in Chapter 6 and the exact computation 
of the optimal policy has been done in the deterministic case in Chapter 7. 
However, when the number of queues in parallel is larger than two, then the 
problem becomes more difficult. One of the reasons for this "three queue gap" 
is that Sturmian sequences in dimension three or more do not exist in most 
cases (see Chapter 2). The only case where this problem has been solved so 
far is when the system is fully loaded and deterministic (see [112]). 

Here, we will deal with the problem of routing customers in several queues 
in parallel using a softer approach. As already presented in [9], we will not try 
to compute the optimal policy, which seems to be a very hard task. Instead, 
we will introduce a partial order on the routing sequences called the regular 
ordering. The main result of the paper is the following: 

If the routing sequence s is more regular than the routing sequence u', 
then the maximal waiting times under s are smaller than the maximal wait- 
ing times under u', for the stochastic order. 

This statement deserves several comments. 

First, the notion of regularity is close the notion of balance (see Chapter 
2) in the sense that a balanced sequence is the most regular sequence possible 
(see Section 14.2.2). An integer sequence is balanced if its partial sums over 
two arbitrary windows of the same length differs by at most one. They have 
been extensively investigated in the past [91, 55] and lead to fruitful results 
in several fields such as theoretical physics [99], combinatorics [86, 29] as well 
as control theory [59, 47] and Part II of this monograph using their close 
relation with bracket sequences. 
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In the previous chapters, it is shown under rather general assumptions 
that balanced sequences minimize several cost functions in discrete event 
systems. This result relies on the following theorem. 

If f n : — ► M are multimodular functions, then the Cezaro limit 
limjy^oo jj J2n=o fn(ui, ■ ■ ■ ,u n ) is minimized by a balanced sequence. 

Second, majorization and Schur convex functions seem to be the right no- 
tions to use when comparing the dispersion of finite sequences [88]. However, 
majorization does not take into account the order of elements of the sequences 
to be compared since it is left invariant by the group of all permutations. 

It is often desirable to take the exact positions into account. For many sys- 
tems, the input sequence (1, 1, 1, 0, 0, 0) certainly induces a different behavior 
than the input sequence (1, 0, 1, 0, 1, 0), which looks more balanced. However, 
these two sequences cannot be compared by using the classical majorization 
technique because the former is a permutation of the latter. To overcome 
this difficulty, we introduce the notion of gap sequences (similar to partial 
sums, already used in [36]) that takes into account the exact positions in the 
sequence. This will narrow the gap between the notion of Schur convexity 
and multimodularity as shown in Appendix 14.6. 

Finally, the usual performance measure in queuing networks is the average 
waiting time. However, in communication models where real time constraints 
are important (such as voice and video traffic) the maximal waiting time (or 
sojourn time) are more important than the average. In Section 14.3, we show 
that for FIFO stochastic event graphs, the maximal waiting time (or sojourn 
time) is regular preserving. As for computational issues, finding good allo- 
cation patterns (i.e. very regular sequences), is possible via a mathematical 
programming problem with convex objective functions as in Corollary 13. A 
similar procedure for minimizing the average waiting time has been used by 
Combe and Boxma [43] and in [36] for different performance measures. 

The rest of the chapter is organized as follows. In Section 14.2, we in- 
troduce the notion of gap sequences and balanced sequences and the regular 
ordering. 

In Section 14.3, we introduce a model of a controlled queue with periodic 
inter-arrival sequences and stationary service times. We show that the maxi- 
mum waiting time is regular preserving with respect to the inter-arrival time 
as well as with respect to the service times. Section 14.3.6 extends the re- 
sult to a routing problem between several queues. Finally we show in Section 
14.4, that the transmission rate on a link with redundancy is also regular 
preserving. 
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14.2 Preliminaries 

This section is devoted to the definition of gap sequences, balanced sequences 
and to the introduction of the notion of regularity. 

14.2.1 Gap Sequences 

We again use the notation N = {0, 1, 2, • • • } for the set of all non- negative 
integers and we consider the set P(T,n) 2]P@P(T, n) of all non-negative 
integer sequences u = («o,«i, ■ ■ ■ , «r-i) of size T which sum exactly to n. 
Let u S P(T,n). We call (p«(0),p„(l), • • • ,p u (n)) the positions in u of all its 
partial sums. 

clef clef 
Pu(0) = and p u (i) = inf{j| ^ u k > i}, Vi > 1. 

fc=0 

We define the vector d l (u) of the gaps of order i > 1 in the sequence it, by 

dpf 

VI < j < n, dj.(«) = Pu ((j + i) mod n) - p u (j). (14.1) 

When the sequence u is in {0, 1} T , then the gaps are never null and measure 
the size of the number of elements in the sequence u between the ones. This 
is illustrated in Figure 14.1. 

4 = 6 



4 = 7 

• - 

4 = 5 



u: 101001000 

- — — - 

4=2 4 = 3 4=4 
Fig. 14.1. Gaps of order one and two for a sequence u in P(9, 3). 



14.2.2 Balanced Sequences 

We repeat here some of the material presented in Chapter 2. 
Definition 22. A sequence u is balanced if for all j, k and n, 



j+n 



< i. 
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A finite sequence u is balanced if the infinite sequence it 00 is balanced. 

Theorem 101 ([91]). If a sequence u in P(T,n) is balanced, then there 
exists 6 e [0, 1] such that for all j > 0, 

% = Lt? + i)^ + 0J-L^ + 0J- 

Proof. The proof follows from Chapter 2 adapted to the finite case. q 

If u is a balanced sequence then the associated gap sequences have the 
following property. 

Lemma 80. If u is a balanced sequence in P(T,n), then for all dj(u) £ 

Proof. By definition (everything is implicitely done modulo n) dj = p(i + 
j) — p(j). By definition of p(j), we have: p(j) = inf{fc| [(k + 1)^ + 6\ > j}. 
Therefore, [(p(j) + l)f + 6\ >j and + < j- This implies that 



Similarly, we have: 



Tl Tl 

j + i ~ jT < P(j + i) f + 8 < j + i- 



As for dj , we get by subtracting the two previous equations 

n J n 

□ 

The following lemma is the converse of Lemma 80. 

Lemma 81. Let u be a sequence such that for all i, all the gaps of order i, 
dj, differ by at most one. Then, u is balanced. 

Proof. Assume that u is not balanced. There exist two intervals of the same 
length (say /) w\ and W2 such that the sum in w\, denoted n\ is larger than 
the sum in w 2 , denoted plus two: n\ > 712 + 2. This means that there 
exists ji and j'2 such that d™ 1 < / and d™ 1 > I + 2. rj 

Lemma 82. A sequence u € P(T,n) is balanced if and only if d}{u) is bal- 
anced. 
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Proof. The "only if" assertion: we consider the sequence d}{u). Let w\ = 
{i, ■ ■ ■ , i + k} and W2 = {j, ■ ■ ■ ,j + k} be two windows of length k. Then the 
sums over w\ of the sequence d x (u) are of the form d\{u) + ■ ■ ■ + d} +k (u) = 

d\ 1 { u ) an d the sums over w 2 are dj(u)-\ \-dj +k (u) = d^ 1 ^). By Lemma 

80, then these two quantities differ by at most one. 

The "if" assertion: d}{u) is balanced means that there exists 8 such that 
for all j, d){u) = [(j + l)^+e\ - [j I Therefore, d) = [{j+i)l+9\ - 
[j ^ + 0\. This means that 

1 < d) < — + 1. 

n J n 

Therefore, for any order i, the gaps of order i differ by at most one. By 
applying Lemma 81, u is balanced. □ 

Remark 36. Note however, that the gaps of order higher than one in a bal- 
anced sequence may not be balanced. For example, if d}{u) = (2, 3, 3, 2, 3, 3, 3) 
(which is balanced), then d 2 (u) = (5,6,5,5,6,6,5) which is not balanced 
because it contains two consecutive 5 (which sum up to 10) and also two 
consecutive 6 (which sum up to 12). 



14.2.3 Regularity and Schur Convexity 

If x e W 1 , then we denote by (£[i],£[2], • • • , x [n\) t ne components of x ar- 
ranged in decreasing order. For x,y € R n , we say that x (resp. strictly) 
majorizes y (denoted x >y) (resp. x y y) if for all 1 < m < n — 1, then 

mm n n 

X \A ^ »M and X H = »M ' 

i=l i=l i=l i=l 

(resp. with strict inequality for at least one to). 

A function / : E™ — ► ffi is (resp. strictly) Schur convex if x >z y => 
f{%) > f{y) (resp. x y y => f(x) > f(y)). For more details on the theory of 
majorization, see [88]. 

Definition 23. If u,u' € P(T,n), then u' is more regular than u (denoted 
u' < s) if for all order i, d z (u') < d z (u). 

Lemma 83. The balanced sequences are the minimal sequences in P(T,n) 
for the regularity order. Moreover, ifu is balanced, then for all u' € P(T, n), s < 
u'. 

Proof. This result is a consequence of Lemma 81. Indeed, if n divides iT, 
then for a balanced sequence a, dj(a) = which is minimal. If n does not 
divide iT, then for the balanced sequence, rfj(c) e {|_7T_I' T^rl} an d is also 
minimal. n 
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Balanced sequences in P(T, n) will also be called the "most regular" se- 
quences in P(T, n) in the following. 

Example 12. Let us compare three sequences in P(6, 3): w\ = (1, 1, 1, 0, 0, 0), 
W2 = (1, 1, 0, 1, 0, 0) and wz = (1, 0, 1, 0, 1, 0), using the regular ordering. The 
respective gap sequences are: 

d^wi) = (1. 1. 4 ) dl M = (1, 2, 3) d^ws) = (2, 2, 2), 
d 2 ( Wl ) = (2,5,5) d 2 (w 2 ) = (3,5,4) d 2 (w 3 ) = (4,4,4), 
d 3 { Wl ) = (6,6,6) d 3 (w 2 ) = (6,6,6) d 3 {w 3 ) = (6,6,6). 

Using majorization, it is not difficult to check that 

d 1 ^) >- d 1 {w 2 ) >- d}{w 3 ), 
d 2 (wi) >- d 2 (w 2 ) >- d 1 ^), 
d 3 (wi) = d 3 (w 2 ) =d 1 {w 3 ). 

According to the definition of the regular ordering, this means that W3 < 
W2 < w\. Actually, W3 is balanced and is the most regular sequence in P(6, 3). 

Definition 24 (regular-preserving functions). A function f : N 1 ^ — ► ffi 
is regular-preserving (r.p.) if x < y =>■ f(x) < f(y). 

Lemma 84. A function f : W — > M is regular-preserving if and only if it 
can be written under the form: 

f(u) = F(P 1 (d 1 ( W )),P 2 (d 2 ( W )), • • • , H n {d n {u))), 

where F : W 1 — > ffi is increasing in all coordinates and Hi : W 1 — ► M, i = 
1 • • • , tj are Schur convex. 

Proof. The only if assertion is a direct consequence of Lemma 83. As for the 
if assertion, then / is necessarily Schur convex in all d\ Now, the combination 
of all these arguments for all i shows that / is of the form given in the lemma. 

□ 

Note that if / is regular preserving on the set P(T, n) then it reaches its 
minimal value for the balanced sequences. 

Remark 37. (Regularity without using the gap sequences) 
The definition of regularity, as well as regular preserving functions, can be 
done in a similar fashion by using the partial sum sequences of the original 
sequence u instead of the partial sums of its order one gap sequence. 

All the results given in this chapter are still true in this new framework. 

The passage from one framework to the other is simply done through 
Equation (14.1) and Lemma 91. 
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14.3 Application 1: Maximal Waiting Times in Networks 

In this section, we show that the maximum waiting time in a queuing system 
with general stationary inter-arrival and service times is a regular preserving 
function. 

14.3.1 The D/D/l Model 

First, we consider a slotted D/D/l queue where the arrival sequence u is 
periodic with n customers arriving every T time slots (each of unit length). 
Therefore, u £ P(T,n)°°. By definition, the number of customers arriving at 
time i is Wi. The i-th customer arrives at time Pi(u) and brings a workload 
of a in the system. 

def 

We will also denote by C = na, the total load brought by n consecutive 
customers. 

We finally require that the queue is initially empty and that the customers 
are served in a FIFO order (this last assumption can be relaxed to a non- 
idling server assumption in some cases). 

Now, let us consider the waiting time Wi of the ith customer that enters 
the system. We have 



We define M(u, a) = max; Wi(u, a). 

Theorem 102. The function M(u) is regular preserving. 

Proof. First, let us remark that if C > T, then Wi — ► oo when i — ► oo. 
Therefore M(u,a) = oo. In the rest of the proof, we will only consider the 
case where C < T. In that case, note that after a transient period of length n, 
the waiting times are periodic: Wi +n = W n , \ii>n. Also note that since the 
system is initially empty, the maximum waiting time is not reached during 
the initial transient period. So we have 




k=j 



def 



290 14 Regular Ordering 

i-l 



M(u, a) = max max {^(a - d\(u)) + a)}, (14.2) 

i=n-\-l j=i—n ' * 



k=j 

i 

— max max {(y"V) — d)~ j (u)}, 

2=7i+l j=i—n ' J J 
k=j 

i 

= max max {(V a) + <%~ i+i {u) - T}, (14.3) 

2 = 71+1 j = i — 71 * 

k=j 

71 + 2 

= maxmax{( > a) + d 3 Au) — Tj, 
i=i j=i 1 

k=j+i 

= maxmax{(TJ - j + l)a + £{u) - T}. (14.4) 
j=i i=i 

Equation (14.2) follows from the fact that C <T. Equation (14.3) follows 
from Equation (14.13). Changing the order of the two max operators yields 
Equation (14.4). 

Now, the function d? i-> maxj l =1 {(n — j + l)a + d\ — T} is Schur convex 
for each j and the function M is increasing in d? for all j, and thus satisfies 
the conditions of Lemma 84. This means that M is regular preserving. q 

Corollary 12. Among all inter-arrival sequences in P(T,n), the balanced 
sequence minimizes the maximum waiting time in the system. Moreover, for 
two sequences u,u' € P(T,n), M(u,a) > M(u',a). 

Remark 38. The function M(u) is multimodular (see Appendix 14.6). This 
is a direct consequence of Corollary 13 there. 

14.3.2 Characterization using regular preserving functions 

The function M when C = T plays an important role. Indeed, we have the 
following result giving yet another characterization of balanced sequences. 

Theorem 103. If C = T , and given u £ P(T,n), if M(u) < M(u') for all 
u' e P(T,n), then u is balanced. 

Proof. When C = T, then a = T/n. Equation (14.4) becomes 
M(u) = maxmkx{-jT/n + dUu)} +T/n. 

i=l j=l 

Note that jT/n is the average value of d^u). Using Lemmas 80 and 81, 
u is balanced is equivalent to the fact that M(u) < 1 + T/n. q 
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14.3.3 The average waiting time is not regular preserving 

In this section, we will show that unlike for the maximal waiting time, the av- 
erage waiting time in a stable D/D/l queue is not regular preserving through 
an example. 

Consider in P(16, 5) the sequences u and it' with the respective one-order 
gaps, 

d\u) = (4,4,3,1,4), 
dV) = (4,4,2,2,4). 

It is immediate to check that u > it'. However, if one computes the average 
waiting time in both cases, when the load brought by each customer is equal 
to 3, one gets: 

W(u) = H < ^ = W(u'). 
5 5 

Remark 39. The function W is multimodular (see theorem 73). Therefore, it 
is minimized by a balanced sequence in P(16, 5): W(u) = 3 if u is a sequence 
such that d}{u) = (3, 3, 3, 3, 4). 

14.3.4 The G/G/l Model 

Here, we generalize the previous model. The deterministic arrival process is 
replaced by a stochastic process with stationary interarrival times: {ri}i G N. 
The load carried by the fc-th customer is denoted by a k - The sequence 
{o"fc}fc e N, is stationary and independent of the arrival times. We denote by 
6i(u) the time elapsed between the arrival of customer i and customer i + 1 
for sequence it, 

P«(«+l) 

6i(u) = ^ Tj, Vi = 0, ■ ■ ■ , m — 1. 

j=Pu(i) + l 

Using Lindley's equation, the waiting time of the i-th customer satisfies 

i-l 

Wi(u) = max{y^(q fc - 6 k (u)) + di}. 

k=j 

If we consider the maximal waiting time within on period, starting with 

customer m + 1, we define ^„(tt) ^= maxj l : j" 7 ^ l +1 Wi(u). 
In a similar fashion as with the D/D/l case, we define 

$r,m(u) d = .max . max {^(a fc - 6 k (u)) + ctJ. 

i=m-\-l j=t—r-\-l ' * 
k=j 
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Intuitively, is constructed by going back in time for r customers only, 
while #™ goes back to time 0. 

Note that if the queue has emptied during the last r arrivals, then both 
quantities coincide. 

We assume that the queuing system is stable, that is TE{ji) > nE(a 1 ) 
(this generalizes the previous assumption, C < T to the stochastic case). 
Under this stability condition, the queue empties almost surely in finite time. 
Therefore, taking r large enough will ensure that and #™ will coincide. 
More precisely, 



" 0)| — a^«, 0, a.s. (14.5) 

in % I \ , 

x max < > ( 

+ 1 j=i-an+l I *-~> 

max max <y^<Tfc+ 6k(u) - 6A1 

= an+l j=i-an+l I ^ ^ 



Now, we have 



*"n,an(u) = . mffi <( V(<7fc - <5fc(lt)) + <7. 

i—an+1 j=t— an+1 I r ' 
^ k=j 



n an I 

= max max < 
i=l j=i ! 



i-i i-l 

k=i—an k=i—an 
i-j-an i-\-an— 1 1 

^ + X! &(u) - ^ ^ • (14.6) 

^ k=j+i k=i k=i J 

Lemma 85. Let us assume that u and u' are two sequences in P{T, n) such 
thatu < u' . Then, an (u) < st ^2n ani u ')> where < st is the stochastic order 
between random variables. 

Proof. For a given pair (i, j) in {1, ■ ■ ■ , n} x {1, ■ ■ ■ an}, we know by definition 
of the regular ordering that there exits i' £ {1, ■■■ ,n} such that c^(it) < 
rfj (it'). Indeed, d\ (u) = bT + d\{u) where bn + r = j, r <n. 

Now, we define a coupling of the service sequence as well as of the arrival 
sequence such that (r' k+i , = ak+i and r' k+p ( ^ = T k+Pu ^) for all k € N. Note 
that the coupled sequences have the same probability distributions as the 
original ones because of stationarity and independence between them and 
because the sequences a and r are independent of u and it'. 

Using this coupling, and considering Equation (14.6), we have directly 



i -j-an i-\-an 



P=j+i' P=j+i 



Since during a periods, the total number of arrivals under u and it' is the 
same and using the coupling, we have 
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i+an-l p u (i+an) 

E <^( w ) = E t p 

k=i p=p u (i)-\-l 
p„(i)+Ta 

= E T P 

P=Pu(i) + l 

= E 



% -\-an—l 

Finally, using the fact that rf^(tt) < c£j («'), we also get 



j+i-l Pu{i+j) 

E = E t p 

fe=z + l 

p„(i)+dj(«) 

= E t p 

P=P„(i) + l 
p„,(i')+d ! J ,(V) 

< E < 

P=Pu'( i ') + 1 

j+i'-l 

= E W)- 



k=i' 



Using these three relations, 



{i-\-an i-\-an— 1 

k—j-\-i k=i k=i 

!% -\-an —1 i'-\-an— 1 

E ^+ E W)- E W) 
k=j-\-i' k=i' k=i' 

This is true for all i, and therefore also for the maximum over all possible 
i from 1 to an. Using the fact that under the coupling, the sequences a' and 
t' are in a "typical" situation, we get (u) < st *" n , an (ii')- □ 

Now, the stationary version of the queue is considered. Using a backward 
coupling argument and Lemma 85, we obtain the following theorem. 
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Theorem 104. Let W; be the stationary waiting time in a stable G/G/l 
queue. Then for two sequences u, u' £ P(T, n) u with u < u' , it holds that 

maxWi(n) < st maxWi(u'). 

i=l i=l 

14.3.5 The Event Graph Model 

We generalize the previous model to event graph models introduced in Chap- 
ter 3. We recall briefly how this generalizes the G/G/l queue. 

— Networks assumptions. We replace the single queue by a network made of 
Q servers (transitions) and P buffers (places) forming a FIFO stochastic 
event graph with a single entry. 

— Stochastic assumptions. The service time at node j for the fc-th customer 
is denoted by a\. The sequence {o~ J k }keN,jeQ is stationary and independent 
of the arrival sequence. However no independence assumption among the 
service times is required. 

— Initial state assumptions. We assume that initially, the event graph is input- 
deadlocked (this is a generalization of the empty queue assumption). 

In Chapter 3 we showed that to any stochastic event graph, we can asso- 
ciate a family of matrices, A(£), of size Q xQ. The entry (i, j) in matrix A(£) 
is the maximum over all the sums of the service times of the ^-th customer, 
o\, in all the nodes k on any path from node j to node i with no initial 
tokens, except at node j, and is — oo otherwise. 

let X q (n) be the epoch when the n-th customer completes its service in 
server q. Using the (max, +) notation, where © stands for max and ® for + 
(see Chapter 3 again), the vector X(n) = (Xi(n),--- ,Xq(ti)) satisfies the 
following recurrence equation: 

X(n)=A(n)®X(n-l). 

def 

Let W n be a vector , with its q-th component equal to: (W n ) q = X q (n) — 
Xo(n). {W n )q can be seen as the traveling time for customer n between its 
entrance in the system and its passage in server q. 

W n satisfies the following equation: 

W n+1 = A{n) ® D(-S n ) ®W n ® B{n + 1), (14.7) 

where D{h) is the diagonal matrix with h on the diagonal and — oo everywhere 
else, and B{n) is a vector which describes the input connection. 

Equation (14.7) can be seen as a vectorial form of the Lindley's equation. 
Its solution is similar to the solution of the scalar case. 

71-1 

W n (u) =fl(n)©0Cj(«), 

3=0 
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with 

CM = (A{k) ® D(-6 k (u))) ® B(n - j - 1). 



If we define as previously, $™ = 0™ m " + i ©j=o CjM and 

771 + 71 2 ✓ 2 — 1 x 

*" ra («)= ® ® ( (g)(^(fc)® BUW))®B(«-i-i) 1 

i=7n+l j=i—r-\-l ^ 

we get by a similar technique as in the G/G / 1 case (with a vectorial notation) , 

|*Zn,a»-*?»| O.S. (14.8) 

provided the event graph is stable (see [20]). 

The quantity $™ti an ( u ) can be transformed using the fact that diagonal 
matrices commute with everything. 



an+n i I i—1 

= ® ® l<g)(A(k)®B(i-j-l)®D(-6 k (u))) 

i=an-\-l j=i—an-\-l \ k=j 
a7i+7i i I i—1 i—1 

= ® ® 0(^(AO®B(i-j-l))®0 /?(-&(«)) 

2=an+l j=z an-\-l \ k=j k=j 
7i an / 2+an— 1 i-\-an— 1 

= ®® {A{k)®B{an- j-l))® D(-6 k (u)) 

i=l j=l y fc=j+2 k=j-\-i 
7i an v-i+an— 1 i+an— 1 

= ®®( (j4(A;) ® f?(an - j - 1)) ® £(-&(«)) ® 

i=l j=l k=j-\-i k=i 
i+j-l v 

!>(«*(«)))■ (14.9) 

fc=i ' 

Lemma 86. -Lei us assume that u and u' are two sequences in P(T, n) u such 
that u < u'. Then, <^ n>an (u) < st *S n , an («'). 

Proof. The proof is similar to the single queue case, using a coupling argu- 
ment and the definition of the regular ordering. 

For a given pair (i, j) in {1, ■ ■ ■ , n} x {1, • • - an}, we know by definition of 
the regular ordering that there exits i' £ {1, ■ ■ ■ , n} such that d\ (u) < d\, (u 1 ). 

Now, we define a coupling of the service sequence such that for each queue 
q, o- k q +i , = a q k+i . As for the interarrival sequence, we set r' k+p (( - () = T k+Pu ^) 
for all k e N. 
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Note that the coupled sequences have the same probability distributions 
as the original ones because of stationarity and independence between them 
and because the sequences a and r are independent of u and it'. 

Using this coupling, we have by definition of A(k), 

i+an-l i'-\-an— 1 

(A(k) ® B(an - j - 1)) = (A(k) ® B(an - j - 1)) . 

k=j+i k=j+i' 

During a consecutive periods, the total number of arrivals with u and u' 
is the same. Therefore, we have using the fact that D(—6k{u)) are diagonal 
matrices, 



i+an— 1 Pu(i+an) 

D(-6 k (u))= D(-t p ) 

k=i p=Pu(i) + l 

P<i(i)+Ta 
= £>(-T p ) 
P=P„(i) + l 
p u ,(i')+Ta 

= D{-r' p ) 

P=Pu' (*') + ! 
i' -fan-1 

= /?(-«;(«)). 



As for the last term, 

j+i-l Pu{i+j) 

£>(&(«)) = D(t p ) 

k=i p=Pu(i) + l 

P„(i)+dj(r0 

= ^(r P ) 

p=p„(i) + l 
Pu'(»')+4( V ) 

< AW) 

p=p«'( i ')+ 1 

= J 2?(^(«')). 

These three relations are true for all i and j, therefore, considering Equa- 
tion (14.9), $2 ntan (u) < st *?„,„.(«')■ n 

Finally, we obtain the following extension of Theorem 104. 
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Theorem 105. In a stochastic event graph, the maximum traveling time 
M(u) over one period of the admission sequence is a regular preserving func- 
tion (in the stochastic sense). 

14.3.6 Routing Problem 

Now, we consider a network of K identical queues in parallel. Customers enter 
a system composed by K queues in parallel. The routing of the customers is 
controlled by a sequence of vectors {W;}, with Wi £ {0, 1} K and 1 < i < T: 
W- = 1 means that the i-th customer is routed to queue j. Note that u is a 
feasible admission sequence as long as J2j = 1 for all i. 

Customers arrive in the system at each time unit and each customer brings 
a constant load of a. 

Figure 14.2 displays the model we are considering. 




Fig. 14.2. Routing of customers in a multiple queue system 



The Constrained Case We consider periodic routing policies U satisfying 
the following constraint: 

Vl<j<K, Y^Ul=nj, (14.10) 

i 

where rij are fixed with J2j n j = T. In other words, u* £ P(T,rij)°°. We 
denote this set of policies by A(n\, • • • , %). 

Now, we define I(u) to be the maximum waiting time of any customer in 

the system, for U € A(ni, ■ ■ ■ ,uk)- I(U) maxj^ M^tP), where Mj is 
the maximum waiting time in queue j. 

We say that the vector (ni,-- - ,nx) is balanceable if there exists U € 
A(ni, ■ ■ ■ ,uk) such that for all j, is balanced. See Chapter 2 for a more 
detailed discussion on balanceable vectors. From Lemma 83 and 102, we get, 
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Lemma 87. // (ni,--- ,tik) is balanceable then I{U) is minimized by a 
routing policy U such that each is balanced. 

Proof. This result is a direct consequence of Lemmas 83 and 102. rj 

This result is similar to some extend to the results proved in Chapter 6 
where multimodularity of the waiting times in each queue is used. However, 
we get a new result here which follows directly from the previous lemma and 
which holds for all (ni, ■ ■ ■ , %) (i.e. not necessarily balanceable). 

Theorem 106. IfV,U £ A{m, ■■■ , n K ), such that for all j, V j > U j , then 
I(V) > I(U). 

The Unconstrained Case Now, we do not fix the number of customers 
sent to each queue. 

We want to consider all admission sequences U in the set 

A = [j A(m, ■■■ ,n K )- 

Lemma 88. The function I(U) is minimized on A for some balanced U £ 
A(n lr -- ,n K ), with n 5 £ {[T/K\, \T/K]},for all j. 

Proof. First, note that if nj e {[T/K\, \T/K]}, for all j, then (m, ■ ■ ■ ,n K ) 
is balanceable. Let b be a vector with such a property. Second, note that 
if not all rij € {[T/K\, \T/K]}, then at least one of them is larger than 
\T/K~\. Finally, note that if U is a balanced sequence in P(T,n) u and if 
U' e P(T, n'Y, with n' > n, then M(U', a) > M{U, a). 

Now, the proof goes like this. By Theorem 106, I is minimized on A(b) 
by a balanced sequence U(b). Moreover, there is some j such that I(U(b)) = 
Mj(U j (b)), with rij = \T/K]. For any U which is not in A(b), then there 
exists k such that rik > \TjK~\. 

Now, I(U) > M k (U k ) > Mj(U j (b)) = I(U(b)). This finishes the proof n 

14.3.7 Computational Problems 

For the constrained problem, we can compute the best sequence using a pro- 
cedure similar to the one used in [36] with quadratic programming techniques. 

As for the unconstrained case, from Lemma 88, it is easy to see that when 
T/K € N, then the best routing policy (which minimizes the function J) is 
the round robin policy. 

However, when T/K N, then little was know before. Here, we can come 
up with the optimal routing sequence using the following procedure. 

First, we compute the optimal sequence b = (m, ••• ,nu) using Lemma 
88. This sequence is unique up to a permutation. Note that by symmetry 
of the cost function, all those permutation will perform the same. Moreover, 
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since all balanced sequences with the same slope in the routing sequence are 
shifts of each other, then for all j such that rij = \T/K], all the waiting 
times Mj(U^(b)) are equal. 

The construction of U^ib) for any j can be done by any known method to 
construct balanced sequence, as for example, by using the bracket sequence 
formula given in Theorem 101. 

14.3.8 A routing example 

Let us assume that we want to route customers into three parallel event 
graphs E\,E 2 ,E 3 with different service times. For some reason, the propor- 
tions of customers sent to E\,E 2 ,E 3 must be 1/2, 1/3 and 1/6 respectively. 
This is a case where the routing policy is constrained to stay in the set 
A(3, 2, 1) with a period equal to T = 6. 

The proportions 1/2,1/3,1/6 are not balanceable. Therefore, it is not 
possible to find a routing sequence which is balanced for each event graph. 
However, we can use the regular ordering to compare several policies. 

Let U = (1, 1, 1, 2, 3, 2) and U' = (1, 1,2,1, 3, 2). 
If we consider the sequence of customers sentinel we get U\ = (1,1,1,0,0,0) 
and U{ = (1, 1, 0, 1, 0, 0). In Example 12, we showed that U[ < Ui. 
If we consider the sequence of customers sent in E 2 we get U 2 = (0,0,0,2,0,2) 
and U' 2 = (0,0,2,0,0,2). The sequence U' 2 is balanced. Therefore, U' 2 < U 2 . 
Finally, if we consider the sequence of customers sent in E 2 we get U3 = 
(0, 0, 0, 0, 3, 0) and U' 3 = (0, 0, 0, 0, 3, 0), which are equal. 

Since the sequence U' is more regular than U for all the event graphs, 
then the maximal waiting time under U 1 is smaller than the maximal waiting 
time under U (in the stochastic sense). This is true for all the distributions 
of the service times in the three event graphs. 

However, if we consider U" = (1,3,1,2,1,2) then U{' < U[ but U' 2 < 
U 2 . The routing sequences U' and U" are not comparable for the regularity 
ordering. For some distributions of the service times in the three event graphs 
then U' will be better than U" and for some other distributions, U" will be 
better than U'. 



14.4 Application 2: Assignment to queues with no 
buffer with redundancy 

In this section, we present an example of optimal control of admission into a 
single buffer queue with redundancy. 

- Transmission opportunities occur at time %,Ti,T 2 , At each trans- 
mission opportunity, a controller can decide to actually transmit a packet, 
or not. If transmission occurs, we assume that it is instantaneous. Define 
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- At a down link node in the network a packet that has been transmitted 
is buffered in a single buffer till its service there is completed. The duration 
of the service is exponentially distributed and it has expectation fi^ 1 . We 
assume that the delay between the transmitter and the down link node is 
zero (the results of this section do not change if we take this time to be any 
other constant period). 

- If a packet is transmitted before the previous packet has completed its 
service time then it is lost. 

A model that can be used to solve this problem was introduced and stud- 
ied in Chapter 8. Here, we further focus on the following problem arising from 
higher layer network considerations. We consider the problem of redundancy 
of packet transmission: each packet is retransmitted at k consecutive trans- 
mission opportunities, which we call a frame, so as to decrease the probability 
of losses. Thus, only if all the k packets in a frame are lost then the frame, 
or equivalently the original information packet, is considered to be lost. 

Our goal, roughly speaking, is to obtain a control with two objectives: on 
one hand it maximizes the average throughput, and on the other it minimizes 
the loss probabilities of frames. Since the throughput is the acceptance rate 
minus the losses, the problem can be formulated as maximization of the 
average number of accepted packets, on one hand, and minimizing the loss 
probabilities of frames, on the other. 

We now formulate more precisely our problem. We define a control policy 
as a sequence {ui}, i £ N, where Ui = 1 if a transmission is scheduled at 
the ith transmission opportunity, and it is otherwise. We consider here the 
case where the controller has no information on the state of the buffer (nor 
on which packet is the beginning of a frame). 

Define D(n) = J27=o u i- The actual nth transmission occurs at time p u {n) 
(see Definition 14.1). We define the process X n to be the number of packets 
in the buffer just prior to time p u {n). Define G(u) as 



where A is a negative constant. We consider the problem of minimizing G 
over all sequences it. 

Note that D(N)/N is the acceptance rate, whereas the second term is 
the average number of frames lost among the first N ones. The cost thus 
contains a term responsible for maximizing the throughput and a term for 
minimizing the loss rates. The parameter A can be viewed as a Lagrange 
multiplier related to a constrained problem; indeed, in Chapter 8 a similar 
problem (for the case k = 1) is studied where the loss rate is to be minimized 
subject to a constraint on the rate of acceptance of packets. 

Next, we focus on the cost G(u). We shall restrict to policies u that are 
periodic (see Chapter 8 for a justification for doing so). Our main result, 
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stated below, is that G(u) is regular-preserving. (This allows us to conclude, 
in particular, that there exists an optimal sequence which is balanced.) 

Theorem 107. Assume that Q are stationary and that the service times 
are i.i.d. exponentially distributed, and independent of the sequence ( k - Let 
u e P(T,n)°°. Then 

(i) G can be rewritten as 

where d\ is defined in (H-l). 

(ii) G(u) is regular-preserving for n> K . 

Proof, (i) Let S n denote the length of the time interval that started when 
the (n — K + l)th packet was transmitted, and ended when the nth packet 
is transmitted. f n (u) := mm k =o,...,K-i X n -k equals if and only if there has 
been no service completion during S n , and is otherwise 1. Thus, the expec- 
tation of f n (u) equals the probability that an exponential random variable 
with parameter /i is greater than or equal to S n . Using the periodicity of u, 
this yields (i). 

(ii) This follows directly from Theorem 108. Note that since is 
convex in x, g(x) = Ee^ 3 ^ is convex too. rj 



14.5 Appendix: properties of the gap sequences 

This appendix gives several properties of the gap sequences which have some 
interest by their own. 

The two following lemmas are straightforward consequences of the defini- 
tion of the gaps. 

Lemma 89. The gap sequences satisfy the following properties: 



d) + d) +l +■■■ + d) +k = d) +1 , (14.11) 

d{+di + --- + di = iT, (14.12) 

d)+d n 3 -i = T. (14.13) 

Proof. The proof is a straightforward consequence of the definition of the 
gaps. □ 

Lemma 90. Letu andu' be two sequences inP(T,n). Ifu' is a shift ofu or 
a mirror ofu (i.e. Vj, Uj = u'- +k oruj = u'_- +k ), then Vi, {d l (u)} = {<f («')}. 

Proof. The proof also follows directly from the definition of the gaps. rj 

Remark 40. The converse of Lemma 90 is false in general, as shown by the fol- 
lowing counter example 1 ; choose u and u' such that d}{u) = (1, 1, 2, 0, 2, 1, 1, 0) 

1 This counter example was provided to the authors by Jerome Galtier 
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and d}{u') = (1, 1,0, 2, 0, 1, 1, 2). u' is not a mirror nor a shift of u. However, 
for all order i, we have {cP(u)} = {<f (V)}. We will prove in the following 
(see Remark 41) that the converse of Lemma 90 is true in the special case of 
balanced sequences. 

Lemma 91. Ifu£P(T,n), then d}-{u) £ P(n,T) and d 1 (d 1 («)) is a shift of 
u. 

Proof. By using Equation (14.12), then d}{u) £ P(n,T). Now, let u' be 
a shift of u such that jv(l) = and such that d}{u') = d}{u). We will 
show that d 1 (d 1 (u')) = u' . Since p u '{i + 1) = inf {j | J2i=o u 'k — * + 1}' then 
2V (i + 1) - 2V («) = Sfc=o M 'fc = Now we compute 

d\{d\u l )) = #{j\Y j d\(.u l )=i} 

k=0 

= #{j\Pu'(j)-Pu'(0)=i} 

= *{j\Pw(j)=i} = <- 

□ 

We can construct the different gaps by computing in the set Z [X] / (X n — 
1). Indeed, from the set d l (u), we can define the polynomial 

Pt(X) d ^j2di(u)X^\ 

3=1 

Note that by definition, we have P % U (X) = (X^ 1 + ■ ■ -+X + 1)P^(X) mod 
{X n - 1). 

Lemma 92. Let i > 1. The original sequence u can be retrieved from d l {u) 
if and only if i and n are relatively prime. 

Proof. There is a one to one correspondence between u and d}{u). Therefore, 
we rather show that the knowledge of d % (u) enables one to compute d)-{u) if 
and only if i and n are relatively prime. If i and n are relatively prime, then 
the polynomials X n — 1 and X z ~ 1 + ■ ■ ■ + X + 1 are relatively prime. Using 
the Bezout equality, there are two polynomials u(X) and v(X) in Z[X] and a 
non-zero integer fc such that fc = (X^ 1 + ■ ■ ■ + X + l)u(X) + v{X)(X n - 1). 
Therefore, by multiplying by Pl{X), we get kP^{X) = P z u {X)u{X) mod 
(X n — 1). The knowledge of P^(X) induces the knowledge of d}{u), which in 
turn gives all gaps by using Equation 14.11. 

On the other hand, if i and n are not relatively prime, then gcd((X* _1 + 

dpf 

• • • + X + 1), X n - 1) = k(X), and if we denote by q(X) = {X n - l)/k{X), 
then q(X) e Z[X] and qiX^X^ 1 + ■ ■ ■ + X + 1) =0 mod {X n - 1). We 
have Pt{X) = P„ 1 (X)(X i - 1 +... + X + 1) mod (X n - 1) as well as P Z U {X) = 
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(Pl(X) + q(X))(X i - 1 + ...+X + 1) mod (X n - 1). Therefore, two different 
gap sequences of order one are compatible with the gap sequence of order i. 

□ 

Remark 1^1. Note that if for a sequence u, all the gaps of all order i differ by 
at most one and if {cf(u)} = {cf («')} for all i, then u and vl are balanced 
by lemma 81. On the other hand, all balanced sequences are shift of each 
other (see Theorem 101) (It is interesting to see that in this case, all mirror 
transformations are also shifts). This is an answer to the question asked 
in Remark 40 in the case where the gap sequences are all formed by two 
consecutive numbers. 



14.6 Appendix: relations between regularity and 
multimodularity 

It is interesting to see if regular preserving functions have some kind of dis- 
crete convexity properties. In this appendix we will show that all regular 
preserving functions are multimodular in some sense whereas the reverse is 
not true. 

Let ei e Ff denote the vector with all components equal to except the 
i-th component which is equal to 1. Define bi = e^-i — e, for 1 < i < T, 
b T = er-i - eo and T = {61,621 ■ ■ ■ > 6t}- 

Since we deal with sequences which sums are fixed to n, we use an adapted 
version of the definition of multimodularity (see Chapter 12). 

Definition 25. A function f on P(T,n) is multimodular with respect to T 
if for all u £ P(T, n), v, w £ T , v ^ w, 

f(u + v)+ f{u + w)> f(u) + f(u+v+ w), (14.14) 

whenever u + w, s + v,s + v + w . 

This is not the classical definition given by Hajek in [59] , which considers 
arbitrary sequences in f^F. However, both definitions are closely related. One 
can show that the projection of a multimodular function (in the sense of 
Hajek) on P(T,n) is multimodular according to Definition 25. (see Chapter 
12 for a deeper insight on the definition given here.) 

Theorem 108. Let f(u) be a function that only depends on d l (u), which is 
also denoted by Hi(d l {u)). If Hi(d l (u)) can be put under the from Y^j=i 9(d l j{u)), 
where g is a convex function, then it is is multimodular on P{T,n). 

Proof. According to the definition of multimodularity, we need to check the 
inequality (14.14) for all u £ P(T,n). 

First note that for all 1 < k < T, there exists jk such that: 
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d){u + b k ) 



d){u) - 1 
d){u) + 1 
d){u) 



if J = Jfc, 
if J = h + i 



otherwise. 



(14.15) 



Now, for any 1 < fc, Z < p n +i such that jk ^ ji ± i, then bi and bk modify 
different gaps. Therefore, 



f(u + b k ) + f(u + bi) - f(u) - f(u + b t + b k ) 

= (<?(4+2») + »(4») +»(4» - !) + 

fl(4+» + !) +5(4+i( M ) " !) +5(4+2i( M ) + !)) " 

(5(4 +5(4+2i( M )) +5(4+i( M )) + 

g{d) k {u) - 1) + fl(4 +2i («) + 1) + 9{d) k+z {u))) 

= -2g{d) k+i {u)) +g{d) k+i {u) + 1) + g{d) k+i {u) - 1) 



This is positive since g is convex. rj 

The following corollary covers the case when several gap orders are used. 

Corollary 13. If for all i, Hi(d l {u)) = Y^=i 9i{dj{u)), where gi are convex 
functions, then define the function 



where Ci are non negative constants. Then f is multimodular. 

def 

Proof. For all i, the function fi(u) = Hi(d*(u)) is multimodular. A positive 
linear combination of multimodular functions is multimodular. rj 

Remark 4%- Note that the functions fi(u) also form a "characteristic set" of 
regular preserving in the same way sums of convex functions play an impor- 
tant role among Schur convex functions ([88]). 

Remark 43. Functions of the form 14.16 have been used in [36] as a criterion 
to test regularity of a sequence. In that paper, the minimization problem of 
functions of this form is written as under quadratic assignment formulation. 
This makes computations possible for small n and T and can be used for 
optimization purposes. 

Remark 44- Finally, one may remark that regular preserving functions con- 
stitute an essential class among multimodular functions. Indeed, as shown in 
Section 14.3.2, if a given sequence in P(T,n) minimizes all possible regular 
preserving functions, then this sequence in balanced. 



f{u + b k ) + f(u + k) - f(u) - f(u + b t + bk) = 0. 



For all 1 < k, I < T such that jk = ji —i, then, 




(14.16) 
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